-- Logs begin at Fri 2022-09-09 20:46:14 UTC, end at Fri 2022-09-09 21:48:55 UTC. -- Sep 09 21:10:42.902199 dv-cinder-852210-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 21:10:42.909661 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 21:10:42 2022] *** Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: nodename: dv-cinder-852210-15-iscsi Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: machine: x86_64 Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: clock source: unix Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: pcre jit disabled Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: detected number of CPU cores: 4 Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: current working directory: / Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: detected binary path: /usr/bin/uwsgi-core Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: your processes number limit is 47680 Sep 09 21:10:42.913309 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: your memory page size is 4096 bytes Sep 09 21:10:42.913951 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: detected max file descriptor number: 2048 Sep 09 21:10:42.913951 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: lock engine: pthread robust mutexes Sep 09 21:10:42.913951 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: thunder lock: enabled Sep 09 21:10:42.913951 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 21:10:42.913951 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 21:10:42.952590 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: Python main interpreter initialized at 0x555c0f3e69c0 Sep 09 21:10:42.952590 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: python threads support enabled Sep 09 21:10:42.952590 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: your server socket listen backlog is limited to 100 connections Sep 09 21:10:42.952590 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: your mercy for graceful operations on workers is 90 seconds Sep 09 21:10:42.952978 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 21:10:42.953057 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: *** Operational MODE: preforking *** Sep 09 21:10:42.953301 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: *** uWSGI is running in multiple interpreter mode *** Sep 09 21:10:42.953301 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: spawned uWSGI master process (pid: 81422) Sep 09 21:10:42.953511 dv-cinder-852210-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 21:10:42.955039 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: spawned uWSGI worker 1 (pid: 81436, cores: 1) Sep 09 21:10:42.955039 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: spawned uWSGI worker 2 (pid: 81437, cores: 1) Sep 09 21:10:42.955486 dv-cinder-852210-15-iscsi devstack@keystone.service[81422]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 21:10:47.266153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 21:10:47.266153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Sep 09 21:10:47.266960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 21:10:47.266960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Sep 09 21:10:47.267374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 21:10:47.267374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Sep 09 21:10:47.267486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 21:10:47.267486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Sep 09 21:10:47.316725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 21:10:47.316901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 21:10:47.316994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 21:10:47.317084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] command line args: [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 21:10:47.317163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 21:10:47.317330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] command line args: [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 21:10:47.317417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 21:10:47.317500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ================================================================================ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 21:10:47.317604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 21:10:47.317746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.317850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ================================================================================ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 21:10:47.318121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_dir = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_dir = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_source = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] debug = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_source = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.318912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] debug = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.319902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_config_append = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_config_append = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.320915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_dir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_dir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.321929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.322999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.323969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] publish_errors = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] publish_errors = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.324999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.325935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] standard_threads = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] standard_threads = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.326901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_journal = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_journal = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_json = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_json = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_stderr = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_stderr = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_syslog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_syslog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 21:10:47.327878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.327967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.328979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.329872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.330974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.331953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.332993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.333171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.333251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.333941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.334956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.335919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.336865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.337937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.338935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.339861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.340933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.341969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.342995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343228 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.343941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.344979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.345940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.346866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.347978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.348905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.349087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.349747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.349873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.349987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.350887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.351989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.352992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.353960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.external = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.external = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.354907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.password = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.token = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.355870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.password = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.token = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.356960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.357974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.358935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.359919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.360913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.361871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.362962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.363930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.364813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.365989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.366926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.367920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.368871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.369975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.370955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.371916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.372852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.373929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.374964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.375948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.376975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.377918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.378995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.379941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.380970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.381917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.382976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.383961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.384985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.385948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.386881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.387994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.388939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.389977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.390931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.391997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.392966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.393946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.394895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.395834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.396509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.396593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.396753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.396966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.397992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.398950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.399931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.400877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.401845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.402940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.403926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.404962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 21:10:47.405859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.405859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.406346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.406346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 21:10:47.406587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 21:10:47.435149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.435361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.435603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.436070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.436302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.436524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.443966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.444935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.472096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.472323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.474811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.475044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.526239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.526450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.575013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.575241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.576030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.576263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.734382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.734629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.734810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.734959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.735978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.758789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.758956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.759984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.760115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.760244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.760373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.760502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.760502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.771753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.772034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.784656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.784845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.785919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.786071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.786204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.786337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.786468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.787023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.787399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 21:10:47.802660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.802949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 21:10:47.803175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.803306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.803439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 21:10:47.803634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 21:10:47.803769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.803898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 21:10:47.804292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.804928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.813968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.814249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.814522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.814808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.814964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.815247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.815516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.815823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.822388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.823319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.825302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.825590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.825878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.826454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.826995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.827973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.828103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.828233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.828362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.828601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.828868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.851144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.852454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.852627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.852913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.853913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.854205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.857561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.857848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.858134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.858832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.859115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.859394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.864491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.864805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.866054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.866343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.870447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.870775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.871256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.871706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.871995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.872277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.872554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.872827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.878021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.878326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.878617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.878908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.879189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.879466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.879774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.880052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.888952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.889234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.893092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.896622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.896901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.896999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.897261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.900716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.901457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.903886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.904184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.904595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.908090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.908393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.908579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.908676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.908953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.909241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.909540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.911778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.914976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.915279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.915586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.915871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.916195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.916485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.925379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.925670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.925941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.926206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.932192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.932484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.932753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.933018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.933296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.934848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.936407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.936688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.939354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.939664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.939824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.939956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.940241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.940635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.941601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.943753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.944031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.946709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.946993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.947274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.947576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.947976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.952377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.952663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.952944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.953225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.953604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.953888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.954170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.959616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.959902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.960184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.960470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.960849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.961129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.961409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.970058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.972982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.976739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.977482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.978212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.979689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.979975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.980372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.980456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.983247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.984705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.986145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.986428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.986697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.986819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.988165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.991819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.992215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.992505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.992788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.993894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.995347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:47.998227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.998622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.998911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:47.999191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.000587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:48.000880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.001176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.007199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:48.007499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.007814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.008077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:48.008364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.008649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.008930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.009207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.009490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.009766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.010048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.010322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.010610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.010883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.011162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.011428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.011732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.014621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 21:10:48.014906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.015188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.015468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.015768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.016041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.016316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.016598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.016871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.017154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.017426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.017698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.018012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.018282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 21:10:48.047295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555c0f3e69c0 pid: 81437 (default app) Sep 09 21:10:48.050509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5b48e-11c8-4c64-96d9-8d78e792e501 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:48.050798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5b48e-11c8-4c64-96d9-8d78e792e501 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:48.051075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5b48e-11c8-4c64-96d9-8d78e792e501 None None] PATH_INFO: `/v3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:48.053477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555c0f3e69c0 pid: 81436 (default app) Sep 09 21:10:48.055352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Fri Sep 9 21:10:48 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:10:54.870237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:54.870745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:54.870852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:54.874993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:10:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:10:54.881122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:54.881538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:54.881676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:54.952941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: INFO dbcounter [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] Registered counter for database keystone Sep 09 21:10:54.971734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG oslo_db.sqlalchemy.engines [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 21:10:54.977944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writer thread running {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 21:10:55.159600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81437) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 21:10:55.159979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81437) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 21:10:55.217940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:55.219478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:55.344643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:10:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 21:10:55.874092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:55.954532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: INFO dbcounter [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] Registered counter for database keystone Sep 09 21:10:55.976285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG oslo_db.sqlalchemy.engines [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 21:10:55.982435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writer thread running {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 21:10:56.102145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d3072-b631-40b6-a7bb-46edef99df29 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:56.106574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:56.107206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:56.107484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:56.107780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:56.116214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:56.167506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 21:10:56.167871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eea62-36bd-4e01-9231-460f183276a5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:56.177230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Fri Sep 9 21:10:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:10:56.183164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:56.200913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f02bb-e51f-4ee0-afa6-9bb0a7494a11 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:56.205693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:56.206756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:56.207040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:56.207311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] PATH_INFO: `/v3/projects/fc5a8e0f96e744888fe26890d52ecbca` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:56.209787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] RBAC: Authorizing `identity:get_project(project_id=fc5a8e0f96e744888fe26890d52ecbca)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:56.263407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING keystone.common.rbac_enforcer.enforcer [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 21:10:56.263784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5dd29-2989-40b9-8770-cde38326e55b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:56.268248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Fri Sep 9 21:10:56 2022] GET /identity/v3/projects/fc5a8e0f96e744888fe26890d52ecbca => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:10:58.403601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951008-1ba1-4a6a-baf7-d06f7f02ed40 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:58.404178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951008-1ba1-4a6a-baf7-d06f7f02ed40 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:58.404178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951008-1ba1-4a6a-baf7-d06f7f02ed40 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:58.408599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:10:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:10:58.413979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:58.414411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:58.414504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:58.434601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:10:58.436120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:10:58.444364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:10:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 21:10:58.710982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29951008-1ba1-4a6a-baf7-d06f7f02ed40 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:58.722869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29951008-1ba1-4a6a-baf7-d06f7f02ed40 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:58.727076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:58.727695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:58.727974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:58.728245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:58.735701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:58.737461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae86f-b287-4652-a37a-2977bcc75be2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:58.746492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Fri Sep 9 21:10:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:10:58.751702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:10:58.761371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde0967-e292-47b3-8a3b-c43fa0703d35 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:10:58.765666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:10:58.766281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:10:58.766558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:10:58.766827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:10:58.768161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:10:58.769972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c4dac-befd-4e17-b58e-2a5c6a0eff34 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:10:58.771479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Fri Sep 9 21:10:58 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:00.982041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09491-7393-4133-b44e-6c64abdc2719 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:00.982550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09491-7393-4133-b44e-6c64abdc2719 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:00.982660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09491-7393-4133-b44e-6c64abdc2719 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:00.983969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:00.991474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:00.991943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:00.992056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.010511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:01.011703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:01.031778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a446e-d2a4-45f5-b97f-3f88dafe71e1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.032070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a446e-d2a4-45f5-b97f-3f88dafe71e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.032349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a446e-d2a4-45f5-b97f-3f88dafe71e1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.033668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:01.038880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.039238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.039523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.082319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81436) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 21:11:01.082787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81436) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 21:11:01.093961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:01.132289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:01.133892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:01.232989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:01.364719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.388380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:01.467864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cd99d-1f85-47c0-8270-b5e20e7a1136 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.501143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.501691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.501972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.502247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.504050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.514001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:01.523629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:01.524729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:01.525746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:01.532582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.605587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b252a4c-7e74-4f17-bf95-fdf3ba501f73 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.607832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:01.615714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:01.634876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21413-acf6-4780-a78f-e15e6adabf2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.645864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.646736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.647018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.647292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.669469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.670074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.670346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.670616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.671312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:01.671643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:01.673032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:01.684710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:01.690327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.713854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:01.715371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:01.791802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62834-e1cc-4d33-add2-a688b60eca2a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.815743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:01.821348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.824896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:01.825412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.826195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.826541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.826756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.827973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:01.829460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:01.831011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:01.837080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.851765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:01.921124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ee580-6805-4717-ba80-9c1bce3678d7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.940099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4209-fdb7-4e8c-a811-8ee03834a5a0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:01.954282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.955134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.955410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.955711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.974760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:01.976236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:01.978503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:01.979124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:01.979399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:01.979713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:01.980815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:01.982225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:01.983777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:01.991295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:11:01.994919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:01.996196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:11:01.997005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:11:01.997005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:11:01.997005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:01.997005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:02.001771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:02.059370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:02.071594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:02.090033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2e22a-5bcb-4ce3-aa93-3a3785f14bc2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:02.105109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9624-0764-4dd0-ad19-44e96c560887 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:02.125813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:02.126410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:02.126687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:02.126958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:02.127702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.domain.id=default, user.id=18ebcbcfc62641d6bab3f3ca3d6448ae) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:02.128007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:02.129466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72893e68-17bb-4935-b312-445b59dbccdf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:02.136309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Fri Sep 9 21:11:01 2022] GET /identity/v3/role_assignments?user.id=18ebcbcfc62641d6bab3f3ca3d6448ae&scope.domain.id=default&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:02.138429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:02.139038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:02.139314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:02.139624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:02.140278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:02.140576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:02.142017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bac51-2e75-46b1-b62a-c36b0c5341b8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:02.162618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:04.431751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb180a-9146-4846-917a-6e7a025413c4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:04.432767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb180a-9146-4846-917a-6e7a025413c4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:04.433250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb180a-9146-4846-917a-6e7a025413c4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:04.434768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:04.444746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0294-d049-4b53-93f7-c015aed7d62e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:04.445210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0294-d049-4b53-93f7-c015aed7d62e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:04.445304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0294-d049-4b53-93f7-c015aed7d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:04.480566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e4ee0294-d049-4b53-93f7-c015aed7d62e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:04.481777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e4ee0294-d049-4b53-93f7-c015aed7d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:04.564851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:04.773945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf706-f4e8-4ab0-af0e-94d83127cd4f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:04.774236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf706-f4e8-4ab0-af0e-94d83127cd4f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:04.774409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf706-f4e8-4ab0-af0e-94d83127cd4f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:04.775913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:04.781264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2405-70a2-4764-b578-5b24a9969574 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:04.781548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2405-70a2-4764-b578-5b24a9969574 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:04.781819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2405-70a2-4764-b578-5b24a9969574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:04.808494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c32a2405-70a2-4764-b578-5b24a9969574 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:04.809673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c32a2405-70a2-4764-b578-5b24a9969574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:04.861829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:04.896033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 12/25] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:05.011233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a078-711e-48ac-8e51-1b036a540283 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.011595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a078-711e-48ac-8e51-1b036a540283 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.011841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a078-711e-48ac-8e51-1b036a540283 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.013171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 14/26] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:05.028932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.029166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.029418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.056687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:05.057895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:05.060745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a078-711e-48ac-8e51-1b036a540283 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:05.174737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 13/27] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:05.251606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:05.274014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a078-711e-48ac-8e51-1b036a540283 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:05.357161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:05.357734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.358014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.358284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.396458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:05.401783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:05.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:05.403442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 15/28] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:05.409141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:05.427909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:05.449777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699753-7bbc-4af1-9a1d-ba04c2a2b29a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:05.462741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:05.523378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:05.524592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.526117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.526818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.547792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:05.550503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2045aae4-9062-4155-a673-79fb59ddb760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:11:05.580068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:05.580986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:11:05.581795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:11:05.581795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:11:05.581795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:05.584777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 14/29] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:05.586268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e74b7-00c7-4ab0-9f7f-de6a7d2e7cc6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.586809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e74b7-00c7-4ab0-9f7f-de6a7d2e7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.587280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e74b7-00c7-4ab0-9f7f-de6a7d2e7cc6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.589128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 15/30] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:05.593683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e74b7-00c7-4ab0-9f7f-de6a7d2e7cc6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:05.644450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4860fb-f972-4f66-8a99-8db493a3b7fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:05.691362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:05.693176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.693630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.694091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.695059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:05.695819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:05.697704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:05.708235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 16/31] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:05.709366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:05.719194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e74b7-00c7-4ab0-9f7f-de6a7d2e7cc6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:05.772287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:05.773241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.773689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.773872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.774575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:05.774941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:05.776445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5218d-0e11-458c-a9d4-5a48ac2a5837 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:05.789266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:05.796500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 16/32] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:05.797865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.798221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.799307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.823070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:05.824428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:05.835731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38677-93eb-4e10-b380-4ee5219b762e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:05.839128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:05.875817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:05.876888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:05.877388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:05.877885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:05.878757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:05.880776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:05.903658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 17/33] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:05.903658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.047606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab68835-3e4c-4317-aabb-6b573fdb079d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.075160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 17/34] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 21:11:05 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:06.076255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.111864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.112531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.112816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.113095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.116645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.118161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.120131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 18/35] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:05 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:06.128042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.162662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:06.319470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43996235-504b-495a-a1be-7d06f625ccd7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.351007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b5b2-0d7a-4161-8832-f134abf7d498 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.364725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.365764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.366471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.366949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.369182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.371557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.373434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 19/36] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:06.374595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.389634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:06.401889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.403245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.403848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.404602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.405619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.407755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.451644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 18/37] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:11:06 2022] POST /identity/v3/projects => generated 301 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:06.452424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:06.517937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:06.544727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e12919-92c1-4e60-9fed-b7cd3027b217 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.591560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1be0c-2529-4fb2-a0f0-54f2a2a4e4c7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:06.594318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.596255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.596538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.596811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.597510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.599028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7121fa-5d75-49f8-a226-ad91a6e41402 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.646777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:06.647754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.648044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.648327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] PATH_INFO: `/v3/domains/default/users/18ebcbcfc62641d6bab3f3ca3d6448ae/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.650453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=18ebcbcfc62641d6bab3f3ca3d6448ae, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:06.652018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937348-9007-4a71-9e1c-7de04cf7c049 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:06.667511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 19/38] 10.222.0.46 () {58 vars in 1361 bytes} [Fri Sep 9 21:11:06 2022] PUT /identity/v3/domains/default/users/18ebcbcfc62641d6bab3f3ca3d6448ae/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:06.673230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c417e-4a31-47aa-a0ff-733f05c66867 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.673939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c417e-4a31-47aa-a0ff-733f05c66867 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.674645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c417e-4a31-47aa-a0ff-733f05c66867 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.676631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 20/39] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:06.678558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3615d-d693-451e-94ce-2919b57a4edb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.679173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3615d-d693-451e-94ce-2919b57a4edb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.679830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3615d-d693-451e-94ce-2919b57a4edb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.686973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 21/40] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:06.689071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.689671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.690218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.715272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:06.717304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:06.749056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 20/41] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 21:11:06 2022] POST /identity/v3/roles => generated 225 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:06.750667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:06.750960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:06.751237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:06.760119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:06.773877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:06.775568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:06.812568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 22/42] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:06.867294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 21/43] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:07.091850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:07.147797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:07.199829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc191c-4002-4738-ad1b-0b19b3d070dd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:07.229226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:07.239841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:07.240486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:07.240773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:07.241126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:07.242170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:07.243832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:07.245302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 23/44] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:07.250568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:07.261758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e6dca-8854-408a-a7c4-251e8bb8ba69 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:07.290189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:07.300581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:07.301468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:07.301756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:07.302036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:07.302721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:07.304269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac3ac5-39e7-4c8f-bf69-c38a2572c7dc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:07.323251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 22/45] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 21:11:07 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:11:07.369069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3668fe-cdee-4acb-9f4c-2322c2c4e8e9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:07.411052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:07.412094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:07.412610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:07.413055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:07.413959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:07.415685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e4d3b-d0db-4c53-b0e4-8682ba366eab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:07.449285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 24/46] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:11:07 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:11:08.947471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c281c-9ffd-411c-9dd8-52f10b2b2757 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:08.948026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c281c-9ffd-411c-9dd8-52f10b2b2757 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:08.948121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c281c-9ffd-411c-9dd8-52f10b2b2757 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:08.949408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 23/47] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:08.954631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:08.955054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:08.955319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.078271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:09.079470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:09.149452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 25/48] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:09.415481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c281c-9ffd-411c-9dd8-52f10b2b2757 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:09.518877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c281c-9ffd-411c-9dd8-52f10b2b2757 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:09.553648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:09.554276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:09.554553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:09.554821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.557014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:09.569724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-19542607-2f58-4317-8311-45b4ba878387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:09.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:09.576518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:09.577273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 24/49] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:09.583294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:09.601320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:09.616625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fa74-56ec-4a9d-86ec-e9f7e34f3f21 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:09.616915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fa74-56ec-4a9d-86ec-e9f7e34f3f21 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:09.617198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fa74-56ec-4a9d-86ec-e9f7e34f3f21 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.618551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 25/50] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:09.623626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:09.624030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:09.624313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.643160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:09.644411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:09.689863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6790-c933-481e-8e59-65ebd4f75396 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:09.711284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 26/51] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:09.723347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:09.723976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:09.724257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:09.724527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.725198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:09.725498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:09.726869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:09.734744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 26/52] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:09.740558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:09.838739 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f3458-d3a4-47ae-b6f4-403997f50057 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:09.966590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:09.977323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:09.977957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:09.978240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:09.978512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:09.979671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:09.981101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:09.982666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 27/53] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:09 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:09.988943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:09.989615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:10.074534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:10.089871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2eb24-7bfb-4b62-9fed-f9d3b75ee18e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:10.095294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ffba2-e67d-4f16-8ca9-ae5337b8d4eb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:10.122924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:10.123574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:10.123860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:10.124144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:10.125268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:10.126709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:10.128297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 28/54] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:10.128761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:10.130355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:10.130677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:10.130963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:10.132162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:10.133614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:10.134593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:10.135177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 27/55] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:10.140587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:10.196764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:10.239215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8776-19c1-45ff-9820-be905a2c3c87 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:10.240377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c4bf9-966d-4312-a35d-33bec2fb3956 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:10.272998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:10.273192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:10.273606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:10.273707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:10.273889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:10.273991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:10.274143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:10.274214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:10.274800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:10.274892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.domain.id=default, user.id=18ebcbcfc62641d6bab3f3ca3d6448ae) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:10.275175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:10.276273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f1e2a-1bf4-4831-bf5a-3feaf102f20e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:10.276634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d210d0-4cc0-4472-a8e0-6a14c620c0e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:10.282724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 28/56] 10.222.0.46 () {56 vars in 1322 bytes} [Fri Sep 9 21:11:10 2022] GET /identity/v3/role_assignments?user.id=18ebcbcfc62641d6bab3f3ca3d6448ae&scope.domain.id=default&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:10.306292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 29/57] 10.222.0.46 () {60 vars in 1140 bytes} [Fri Sep 9 21:11:10 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:11:12.496040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1088086-8ec0-48f3-9caa-877083f15172 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:12.496557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1088086-8ec0-48f3-9caa-877083f15172 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:12.496640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1088086-8ec0-48f3-9caa-877083f15172 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:12.497935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 29/58] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:12.502989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:12.503423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:12.503628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:12.521561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:12.522760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:12.587059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 30/59] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:12.858133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1088086-8ec0-48f3-9caa-877083f15172 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:12.873508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:12.959701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1088086-8ec0-48f3-9caa-877083f15172 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:12.992021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:12.992644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:12.992927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:12.993202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:12.994311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:12.995795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19733210-9f8b-492f-b5a2-ead3b7840e0e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:12.997347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 30/60] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:13.002830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:13.045117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:13.105487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dec87-9e3b-4512-ad3c-c4e4e57f54dc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:13.137521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:13.138058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:13.138336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:13.138604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:13.139263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:13.140732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737993f-117d-4fc3-a410-aaa995fe3e59 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:13.166892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 31/61] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:13 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:16.489908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7271c-dca2-4e6a-b7db-9261b98a85fd None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.490414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7271c-dca2-4e6a-b7db-9261b98a85fd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.490489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7271c-dca2-4e6a-b7db-9261b98a85fd None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.491827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 31/62] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:16.497368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ac2a1-21cc-4c4c-bc38-ca2d4779195f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.497765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ac2a1-21cc-4c4c-bc38-ca2d4779195f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.497932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ac2a1-21cc-4c4c-bc38-ca2d4779195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.524635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b88ac2a1-21cc-4c4c-bc38-ca2d4779195f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:16.525844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b88ac2a1-21cc-4c4c-bc38-ca2d4779195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:16.616859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 32/63] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:16.694536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81890-3f67-434c-85b1-c5749209e89c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.694842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81890-3f67-434c-85b1-c5749209e89c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.695099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81890-3f67-434c-85b1-c5749209e89c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.696608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 32/64] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:16.701932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.702450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.702920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.733931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:16.735472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:16.763373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8f220-dbf5-4b69-84f5-d5455c5cb0eb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.763997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8f220-dbf5-4b69-84f5-d5455c5cb0eb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.764478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8f220-dbf5-4b69-84f5-d5455c5cb0eb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.764675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:16.766247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 33/65] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:16.774775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:16.775368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:16.775881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:16.796418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:16.798428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:16.827994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 33/66] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:16.890492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 34/67] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:16.935769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.040981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007d0af-46f2-4043-8e21-502f50c856af None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.076538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.077516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.078003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.078479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.089123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4a9f9bf3-e663-4e9c-a20e-62862b61ff5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:17.094340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:17.095179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:17.096005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:17.098813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 34/68] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:17.104740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.122350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.146080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73be32-7114-4ff7-b4be-17fd8c894b7f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.146696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73be32-7114-4ff7-b4be-17fd8c894b7f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.147197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73be32-7114-4ff7-b4be-17fd8c894b7f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.148782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 35/69] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:17.155223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.155775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.156276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.191582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:17.191582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:17.207591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.268279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d710465-bc00-4496-bf5c-06fc8a76aecc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.274638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 36/70] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:17.276398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.309393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.309936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.310475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.310765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.311885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:17.312204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:17.314045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:17.324814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 35/71] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:17.328431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.398987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824c7f6-4a8e-4478-95e6-a424039b5edd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.412707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.435923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.436558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.436837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.437109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.446651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ee092-8665-4253-bc2b-5b59466f103b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.447061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:17.451948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:17.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:17.453776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:17.454786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.466981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.494054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.494690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.494976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.495256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.506162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b318ff20-19d1-44c4-a150-64d0eb9bd043 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:17.511487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:17.512440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:17.513201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 36/73] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:17.513833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaf12d-6716-49fd-810b-b77c400af86f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.514467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaf12d-6716-49fd-810b-b77c400af86f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.514467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaf12d-6716-49fd-810b-b77c400af86f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.515582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 37/74] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:17.516258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaf12d-6716-49fd-810b-b77c400af86f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.568096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061acb79-7269-41ee-a5a4-80bf87646dc8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.603223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.603877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.604160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.604436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.619335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:17.620834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:17.621512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaf12d-6716-49fd-810b-b77c400af86f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.622506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 38/75] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:17.627259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.657938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.663615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.664225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.664510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.664786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.665454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:17.665761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:17.667158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286df6e5-767a-4ced-a398-11d75eda093b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:17.680122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 38/76] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:17.680122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.680122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.680122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.695646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:17.695909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.696886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:17.730900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6fc6-16a4-45f4-8a20-4de6270e62b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.764570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.764996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 39/77] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:17.765187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.765472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.765773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.765893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.766423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:17.766731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:17.768140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:17.780772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 39/78] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:17.781582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:17.889246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22299a43-4e59-4c85-aeab-777407418a03 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.897327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a122861-e028-4e4e-8871-ba518f82b720 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:17.916008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.926773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:17.928486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.929289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.929600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.929957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.936213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:17.936993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:17.937421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:17.937843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:17.940270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:17.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:17.946982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 40/79] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:17.947914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.021429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.023162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.049982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535fd2a-04dc-497c-851d-d346c1cd7d5e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.084435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.085310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.085599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.085880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.087045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.088544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.090125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 41/80] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.090895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.125592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 40/81] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:18.126816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.142052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.155927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.189736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d84b-5b98-47e5-a67c-0a1ab922e249 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.223408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.224081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.224423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.224635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.225761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.227195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.228030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04ad4c-4d52-439c-8653-96682284cb34 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.233428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 42/82] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.233428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.267668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.268285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.268570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.268876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.269544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:18.269853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.271238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.279390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 41/83] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.279390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.328537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9913b-fd44-4533-8259-878079c15aa5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.347810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.361914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.362543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.363357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.363834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.364981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.366428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.368023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 43/84] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.368796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.369659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.380308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5517-cc95-4b10-9dba-cb7d0c7480c9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.413382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.413997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.414285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.414565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.466742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f2f5-ddb8-4da0-8a5b-f445afa9f8e2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.481613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.483075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.499871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.500497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.500775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.501049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.502171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.503653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.505280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 44/85] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.506027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.561558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.572573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.584026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 42/86] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:18.584950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.611130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fe22-2b7a-4c9f-a031-f60a268a9da5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.649524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.650142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.650433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.650713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.683292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a39fc-0526-4f89-88d4-d6533ecbf355 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.715931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.716796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.717422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.717521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.717708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.717985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.719123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.720572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.722289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 43/87] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.723044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.773282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.780567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:18.819041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 45/88] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:18.819920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.821999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba0dc-93ec-4f33-9d60-c0cb8fc40818 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.855880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.856462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.856741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.857013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.857665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.859112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.881519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 44/89] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:18 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:18.882343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.946090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cfc22-b681-465d-ace5-43ac1b0f807f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:18.985813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:18.986735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:18.987028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:18.987304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:18.988668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:18.989045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:18.990570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:18.996763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 46/90] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:18.997640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:18.998802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c757fb-1205-4e6d-b371-cad18cf4e90d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.032900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.036818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.054422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.055375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.056109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.056594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.058347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.060041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.061998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 45/91] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:19.062988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:19.151241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eff8-976f-46d0-939e-0722c2dbd92c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.194638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.195268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.195581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.195873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.204758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5d3ce-3c57-4703-a3ef-a98a64e4493b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.249228 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.250240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.250530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.250810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.254046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.255922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 46/92] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:19 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:19.256706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:19.299634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.302326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.327910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.342296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.393391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10c402-9cb3-4044-aecc-1b43770ae803 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.435044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.435898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.436148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.436431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.437446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:19.437621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.439024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.444815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 47/93] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:19.446461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:19.453807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 47/94] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:19 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:19.454894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:19.600401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414386dd-fee1-4ad3-9c23-ac094f3caf41 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.628207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.644607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2f74-b48f-4bd8-bd64-be961483de09 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.648172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.648800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.649082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.649357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.650108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=18ebcbcfc62641d6bab3f3ca3d6448ae) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:19.650420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.651937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.665188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 48/95] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:19 2022] GET /identity/v3/role_assignments?user.id=18ebcbcfc62641d6bab3f3ca3d6448ae&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:19.688018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:19.710024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.710608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.710895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.711174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] PATH_INFO: `/v3/projects/75c5a82f81e94f918b6738ed3cb6a293` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.716856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] RBAC: Authorizing `identity:get_project(project_id=75c5a82f81e94f918b6738ed3cb6a293)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.719311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcfbe0-9d7e-41c7-8fe1-28742880b737 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.721157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 48/96] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:19 2022] GET /identity/v3/projects/75c5a82f81e94f918b6738ed3cb6a293 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:19.728436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:19.866249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5d4b9-8c20-450b-b5cd-0b91379dac8c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:19.910312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:19.910312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:19.910312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:19.910312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:19.911370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=75c5a82f81e94f918b6738ed3cb6a293, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:19.911370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:19.911556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14540-effa-4014-9d92-91b7c921c880 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:19.917755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 49/97] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:19 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=75c5a82f81e94f918b6738ed3cb6a293&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:22.118688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51c1b-0706-4b0d-a640-bf2d17287c1d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.119429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51c1b-0706-4b0d-a640-bf2d17287c1d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.119429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51c1b-0706-4b0d-a640-bf2d17287c1d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.120781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 49/98] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:22.134949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2873e3-ef0c-4a66-8619-ccce3c53261c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.135495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2873e3-ef0c-4a66-8619-ccce3c53261c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.135495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2873e3-ef0c-4a66-8619-ccce3c53261c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.160791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ed2873e3-ef0c-4a66-8619-ccce3c53261c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:22.162015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed2873e3-ef0c-4a66-8619-ccce3c53261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:22.218069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:22.254929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 50/99] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:22.394821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cce7-8d73-41c1-9703-c73762064fa5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.395044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cce7-8d73-41c1-9703-c73762064fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.395373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cce7-8d73-41c1-9703-c73762064fa5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.396637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 50/100] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:22.413181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8553-02f0-460c-a2d7-ceb08f76caa0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.413467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8553-02f0-460c-a2d7-ceb08f76caa0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.413752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8553-02f0-460c-a2d7-ceb08f76caa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.439368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c22b8553-02f0-460c-a2d7-ceb08f76caa0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:22.441169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c22b8553-02f0-460c-a2d7-ceb08f76caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:22.541791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 51/101] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:22.712577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d51a-599c-421c-a7c8-5b4eaca1e1f6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.712888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d51a-599c-421c-a7c8-5b4eaca1e1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.713139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d51a-599c-421c-a7c8-5b4eaca1e1f6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.714483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 51/102] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:22.719769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:22.720059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:22.720337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:22.747798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:22.748998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:22.840044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 52/103] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:22.889283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d51a-599c-421c-a7c8-5b4eaca1e1f6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.004112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.031849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d51a-599c-421c-a7c8-5b4eaca1e1f6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.083520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.084647 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.085153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.085753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.087304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:23.089163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:23.091198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 52/104] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:23.099568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.107924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.212861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.226872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6eb47b-d7d2-4083-a790-34727431ddf2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.264033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84401b48-94f1-4059-b1df-578d3f09ec9c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.274699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.275370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.275679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.275952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.288246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-451862fa-001f-49ef-b823-813e1b97cb4f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:23.293641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:23.294767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:23.295571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 53/105] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:23.296228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2fd6-f3e6-44c2-b102-6866724d0508 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.296228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2fd6-f3e6-44c2-b102-6866724d0508 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.296350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2fd6-f3e6-44c2-b102-6866724d0508 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.297679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 54/106] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:23.302089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b2fd6-f3e6-44c2-b102-6866724d0508 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.311845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.312877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.313171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.313450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.314117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:23.316942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba47719-befa-4733-844f-369d48f1015c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:23.349162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 53/107] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:11:23 2022] POST /identity/v3/projects => generated 302 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:23.350497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5065a0b-fa65-4e6b-880b-e78c658c8666 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.350794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5065a0b-fa65-4e6b-880b-e78c658c8666 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.351071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5065a0b-fa65-4e6b-880b-e78c658c8666 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.352378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 54/108] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:23.353101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5065a0b-fa65-4e6b-880b-e78c658c8666 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.423500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.431299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b2fd6-f3e6-44c2-b102-6866724d0508 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.470752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.471410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.471793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.472244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.482189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5065a0b-fa65-4e6b-880b-e78c658c8666 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.483696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:23.484643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-bce773ab-147b-46ab-8d74-8fcbde94b415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:23.489670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:23.490638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 55/109] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:23.496651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b08ec-cd4c-49a9-b91f-5480bef9283e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.497444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b08ec-cd4c-49a9-b91f-5480bef9283e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.497444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b08ec-cd4c-49a9-b91f-5480bef9283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.513832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b4b08ec-cd4c-49a9-b91f-5480bef9283e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:23.515132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b4b08ec-cd4c-49a9-b91f-5480bef9283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:23.531260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.532072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.532354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.532625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.533302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:23.533607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:23.535027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1614539-185f-47b4-ba01-92a1a0bc4c6e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:23.543971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 55/110] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:23.545180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.545464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.545819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.568835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:23.570038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:23.594083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 56/111] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:23.595661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde9e6-aa6f-407b-9964-f46c978460c9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.595661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde9e6-aa6f-407b-9964-f46c978460c9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.595980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde9e6-aa6f-407b-9964-f46c978460c9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.601470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 57/112] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:23.601470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fde9e6-aa6f-407b-9964-f46c978460c9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.656560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 56/113] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:23.657874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.698368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.711936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fde9e6-aa6f-407b-9964-f46c978460c9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.750931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.751594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.751879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.752151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.752845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:23.753222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:23.754822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27ed79-be69-4589-a498-1976f421475d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:23.764128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 58/114] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:23.765453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.765728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.766005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.778540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:23.785972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:23.787066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:23.789030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84910a-3cbd-4fea-a2ac-07f143dd2ca8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:23.841235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:23.842565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:23.843082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:23.843640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] PATH_INFO: `/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:23.863001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] RBAC: Authorizing `identity:get_user(user_id=3b09f3dd5e2148f59d5c7bab89e1b4dd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:23.864824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:23.866428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 59/115] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:23.867177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 57/116] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:23.871597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.874998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:23.992175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ebee-7d69-4e5f-bd88-a3c979d5ef78 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.032470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.033438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.033922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.034953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.058754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.061902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.105685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b4ff8-1813-4419-8379-f043c75782ef None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.116164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.117949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.171903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.172660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.172994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.173275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.174437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.176864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.179321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 60/117] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:24.180923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.243820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 58/118] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:24.244507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.304548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.314325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5604b5-032d-405c-8c00-f22f9a66e31e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.351459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.352152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.352437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.352717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.353142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4eb74e-2cfb-4604-a948-867851693992 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.365724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:24.371015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:24.371893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:24.372667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 61/119] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:24.377829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.390429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.393368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.394129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.394477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.394761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.405392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:24.411598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:24.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:24.413235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 59/120] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:24.413862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.503655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d3434-bc7b-46e4-ba75-41ca87754d17 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.526748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543cc74-d72f-41e6-8c65-0d21cb338738 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.541035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.541934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.542747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.542854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.543110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.544335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.545784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.547728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 62/121] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:24.548511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.568864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.569828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.570178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.570453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.571662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.573156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.574918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 60/122] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:24.580323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.634602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.659997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba620bd-38d4-4939-b090-d15caa41964e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.696490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a7cf7-0394-4c63-a806-1af017f91837 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.696775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.697200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.697484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.697759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.708782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:24.713594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:24.714654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 63/123] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:24.715658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.737694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.738307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.738588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.738859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.739518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:24.739886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.742046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.751429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 61/124] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:24.752400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.803024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.824754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7be6-4a7b-42b8-9093-ac90c19a85c5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.863632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.864325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.864604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.864874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.865533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:24.865834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.865930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624096bf-eb4d-43ed-ac2f-2b77e27d9b1d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:24.867311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.870312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:24.883333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 64/125] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:24.883333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:24.908156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:24.908814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:24.909317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:24.909639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/41b5e9ce17744df388fc2899d20c97a1/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:24.911553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, user_id=41b5e9ce17744df388fc2899d20c97a1, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:24.912963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:24.930149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 62/126] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:24 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/41b5e9ce17744df388fc2899d20c97a1/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:24.936522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.021524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a311c-c218-485f-a9ce-69e3ad1b69bf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.070359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e723778-f5c1-41af-b86b-fd0d7b1ecd78 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.071490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.072183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.072472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.072750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.073507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=3b09f3dd5e2148f59d5c7bab89e1b4dd) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:25.073704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.073811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.075627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.083744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 65/127] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/role_assignments?user.id=3b09f3dd5e2148f59d5c7bab89e1b4dd&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=aee3894c6efa40af958645465c848a05 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.084714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.144922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.146165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.146656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.147143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.148102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:25.148877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.151767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.168623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 63/128] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.170774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.179262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.238428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0eb81-184a-49a7-81ef-13bda96b46f2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.293011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.293665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.293953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.294405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.296444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.297937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.300400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 66/129] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.301167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.316460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b63bbf-db43-4509-87e0-5c5dd8b801c2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.352775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.353326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.353610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.353889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.355037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.356510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.358432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 64/130] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.359358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.394059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.467592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de770e-95fc-40eb-8d13-273c722058ce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.498491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d394-d516-419a-bb79-08207df0033d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.510222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.510871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.511150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.511424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.513395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.514830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.519524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 65/131] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.520382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.582424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.608082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.608650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.608933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.609208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.616445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.617933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.619622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 67/132] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.620370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.656026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963edb-01b4-40b4-bf97-f583ece83653 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.665549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.707914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.714281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.714570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.714854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.716169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.717627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.719283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 66/133] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.720092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.793227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddf65-f934-41f7-a000-38d7cc6f67e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.837808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.838500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.838776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.839054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/18ebcbcfc62641d6bab3f3ca3d6448ae/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.839299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43a7c2-0473-40fb-b93b-622888bf8a9e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:25.841232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, user_id=18ebcbcfc62641d6bab3f3ca3d6448ae, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.843501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1e54f-901d-4fa7-ab41-ea8f829d496c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.862950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 68/134] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:25 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/18ebcbcfc62641d6bab3f3ca3d6448ae/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:25.864268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359cbdd-c649-458e-b57d-70fb1a6498bb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.864605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359cbdd-c649-458e-b57d-70fb1a6498bb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.865411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359cbdd-c649-458e-b57d-70fb1a6498bb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.867258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 69/135] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:25.868201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359cbdd-c649-458e-b57d-70fb1a6498bb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:25.881339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:25.882142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.882534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.882929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] PATH_INFO: `/v3/projects/75c5a82f81e94f918b6738ed3cb6a293` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.884201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] RBAC: Authorizing `identity:get_project(project_id=75c5a82f81e94f918b6738ed3cb6a293)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:25.885848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da207b82-5671-4bb7-a9ac-2584b01e4fca None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:25.888145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 67/136] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:25 2022] GET /identity/v3/projects/75c5a82f81e94f918b6738ed3cb6a293 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:25.889461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:25.889907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:25.890285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:25.911224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:25.912624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:25.951908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:25.967963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:26.020799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359cbdd-c649-458e-b57d-70fb1a6498bb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:26.043959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 68/137] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:26.045066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:26.088298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:26.089023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:26.089306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:26.089633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/41b5e9ce17744df388fc2899d20c97a1/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:26.091929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, user_id=41b5e9ce17744df388fc2899d20c97a1, role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:26.093482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:26.111138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 70/138] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:25 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/41b5e9ce17744df388fc2899d20c97a1/roles/da0177264e704fa19ac1889c21e21eb5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:26.249686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a2e96-9a2d-4be6-b382-7f9299611bc0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:26.297583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:26.298256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:26.298541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:26.298819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] PATH_INFO: `/v3/projects/75c5a82f81e94f918b6738ed3cb6a293/users/41b5e9ce17744df388fc2899d20c97a1/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:26.305022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] RBAC: Authorizing `identity:create_grant(project_id=75c5a82f81e94f918b6738ed3cb6a293, user_id=41b5e9ce17744df388fc2899d20c97a1, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:26.306491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:26.327242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 69/139] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:26 2022] PUT /identity/v3/projects/75c5a82f81e94f918b6738ed3cb6a293/users/41b5e9ce17744df388fc2899d20c97a1/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:26.406912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:26.537590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92a46-fa8e-4cb1-bad1-b8cf7c5d5269 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:26.560938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:26.582346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:26.583582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:26.584363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:26.584643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:26.585904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:26.587584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec3331-6144-4f5d-86ca-a72ac6dad876 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:26.589197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 71/140] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:26.594772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:26.657922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:26.776021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b13b5-2372-4447-a8d4-f3f802a2d32b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:26.817444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:26.818381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:26.818874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:26.819348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:26.820255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:26.821912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07058d-3161-4a71-b272-980447d4dc40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:26.852228 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 70/141] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:26 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:27.801709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd39054-bf38-4271-9fdc-7846a5b4928b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:27.802265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd39054-bf38-4271-9fdc-7846a5b4928b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:27.802265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd39054-bf38-4271-9fdc-7846a5b4928b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:27.803832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 72/142] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:27.804716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f3a0-19c7-4e68-b1b2-1a5733f21dba None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:27.805175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f3a0-19c7-4e68-b1b2-1a5733f21dba None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:27.805371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f3a0-19c7-4e68-b1b2-1a5733f21dba None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:27.806833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 71/143] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:27.808962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:27.809258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:27.809534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:27.818285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:27.818578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:27.818863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:27.829998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:27.831262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:27.837978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:27.842043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:27.906071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 73/144] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:27.924864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 72/145] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:28.230408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.370616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c29174-f466-4bdb-9cd1-b9c471bc71fe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:28.392517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:28.416459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:28.417108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:28.417390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:28.417662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:28.429440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:28.434318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:28.435228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:28.436459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 74/146] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:28.442450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.457890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:28.507120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.601012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423bd0f-86eb-4558-b089-aa11a05e6589 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:28.663104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:28.664137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:28.664731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:28.666204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:28.667079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:28.668842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:28.670429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:28.687221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 73/147] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:28.689398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc0e8-b852-4de0-a43e-4924aecb32ac None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:28.692662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.763088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:28.764100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:28.764613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:28.765090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:28.784090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:28.789885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:28.790733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:28.791492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:28.800089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 75/148] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:28.804653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.842497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:28.883942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:28.899109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e561b1d-a60b-4bd1-acc1-a39870f51b10 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:28.972786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:28.973451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:28.973918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:28.974339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:28.975748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:28.977403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:28.979157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 74/149] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:28.992545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:28.999353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2557a30-f151-409a-95a1-1475d78a03a1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.062328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.063174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.063655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.064091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.064917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:29.065494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.067160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf36f2-4aa0-4e32-92b2-9bb6cb862bfa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.081110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 76/150] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:29.082765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4c43-16f7-4c01-b4e4-2e9f3ab8fda9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.083202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4c43-16f7-4c01-b4e4-2e9f3ab8fda9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.083658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c4c43-16f7-4c01-b4e4-2e9f3ab8fda9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.085044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 77/151] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:29.088737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4c43-16f7-4c01-b4e4-2e9f3ab8fda9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:29.196430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6ba71-c377-48b9-9e48-38807889e2b1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.248209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.248831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.249117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.249400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.268115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:29.289861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:29.304380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c4c43-16f7-4c01-b4e4-2e9f3ab8fda9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.336743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.338508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.378234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.378953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.379330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.379728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] PATH_INFO: `/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.384215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] RBAC: Authorizing `identity:get_user(user_id=3b09f3dd5e2148f59d5c7bab89e1b4dd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.385661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcc270-c990-44ec-b312-8808317168bb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.387423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 78/152] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:29 2022] GET /identity/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:29.388735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.389175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.389589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.425161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:29.426564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:29.453531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 75/153] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:29.454536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:29.508453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:29.561186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 79/154] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:29.562077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:29.572865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c177e-3e0c-4e02-a195-c05ce749482c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.610398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.611300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.612418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.613287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.614640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.616298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.618319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 76/155] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:29 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:29.630437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:29.750952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:29.762509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a327d7f-25f3-4e02-aad0-6764f9c365cc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.776762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad366744-5630-4b83-a6ba-da34792bf8ec None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:29.793850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:29.827321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.828574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:29.829446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.830351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.831261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.831468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:29.831997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:29.832420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/3b09f3dd5e2148f59d5c7bab89e1b4dd/roles/aee3894c6efa40af958645465c848a05` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:29.834581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.836709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.837538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, user_id=3b09f3dd5e2148f59d5c7bab89e1b4dd, role_id=aee3894c6efa40af958645465c848a05)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:29.839118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:29.841122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 80/156] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:29 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:29.852908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:29.868846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 77/157] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:29 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/users/3b09f3dd5e2148f59d5c7bab89e1b4dd/roles/aee3894c6efa40af958645465c848a05 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:29.992415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de607d9-71e2-42d2-88e0-90c6004272fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:30.036846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:30.037469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.037745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.038070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.038821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:30.039126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:30.040631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:30.049255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 81/158] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:29 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:30.074047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:30.227037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7af340-ac92-4b72-99d0-210e66f25d6f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:30.266126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:30.267112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.267643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.268605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.281263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:30.287869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:30.288997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:30.289982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:30.292035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 78/159] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:30.301362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:30.356176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:30.463476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618411a-96b4-4d43-b71c-25e5761fde2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:30.512276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:30.512802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.513076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.513342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.514008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:30.514305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:30.515691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529de2-4058-4e13-ac01-d642b646ca9e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:30.525647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 82/160] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:30.531796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:30.582767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf1d76-7f72-451e-9fd5-482d56faebd2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.583069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf1d76-7f72-451e-9fd5-482d56faebd2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.583324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf1d76-7f72-451e-9fd5-482d56faebd2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.585883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 83/161] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:30.586928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:30.589994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e730-ab66-4bd2-8611-18b69a168590 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.590282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e730-ab66-4bd2-8611-18b69a168590 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.591888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e730-ab66-4bd2-8611-18b69a168590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.629102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9fd2e730-ab66-4bd2-8611-18b69a168590 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:30.630358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9fd2e730-ab66-4bd2-8611-18b69a168590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:30.703491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2a145-8b6c-4c1e-a34e-ffa154af2b93 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:30.747667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:30.748728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.749263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.749733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] PATH_INFO: `/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.751136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] RBAC: Authorizing `identity:get_user(user_id=18ebcbcfc62641d6bab3f3ca3d6448ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:30.754050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:30.755985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 79/162] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:30 2022] GET /identity/v3/users/18ebcbcfc62641d6bab3f3ca3d6448ae => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:30.762738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:30.769824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 84/163] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:30.771390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1b05a-2332-46ed-aa2f-3e821134d1b3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.774301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1b05a-2332-46ed-aa2f-3e821134d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.774728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1b05a-2332-46ed-aa2f-3e821134d1b3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.778875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 85/164] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:30.785906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:30.786389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:30.786814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:30.822423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:30.825990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:30.858938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:30.974654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 86/165] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:30.977819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:30.982765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380056cf-44da-4f85-97e2-0220c4e78dfe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:31.024847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:31.025697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:31.026131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:31.026542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:31.109058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:31.110771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:31.218196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:31.223739 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 80/166] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:31.229125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:31.375386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea532b8c-7979-4ecd-ba0b-ac6f8f24dce9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:31.424302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:31.425495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:31.426014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:31.426521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:31.427936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:31.429605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:31.431829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 87/167] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:31.437479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:31.460179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:31.491205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:31.637866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e0347-b87a-4bc8-87b5-b6dfeadc87e4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:31.670578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9501a44-0821-490a-a1c2-173d25dfb972 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:31.712030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:31.713941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:31.717334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:31.718888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:31.719939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=18ebcbcfc62641d6bab3f3ca3d6448ae) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:31.720676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:31.724616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:31.736372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 81/168] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/role_assignments?user.id=18ebcbcfc62641d6bab3f3ca3d6448ae&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:31.738821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:31.754746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:31.755811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:31.756304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:31.756928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:31.780673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:31.787129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:31.788112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:31.788946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:31.799102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 88/169] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:31.806310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:31.865335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:31.914064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f9367-be15-4cbb-a98e-09330f91f06e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:31.959800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:31.961076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:31.961638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:31.962828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:31.963310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:31.977451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:31.983569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:31.984951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:31.985742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:31.989268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 82/170] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:31.995806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:32.007822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f8ae1-1d51-4021-984e-150c8d2c0b8b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.076344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.076976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.077258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.077526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.078201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:32.078498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.080186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3c9e0-d269-4238-a1a1-a7fe2a00a414 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.093852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 89/171] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:32.095072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed03b-77f1-4655-bb0c-4fbf84333280 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.095353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed03b-77f1-4655-bb0c-4fbf84333280 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.095649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed03b-77f1-4655-bb0c-4fbf84333280 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.097051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 90/172] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:32.099312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ed03b-77f1-4655-bb0c-4fbf84333280 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:32.129560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ffdef-a82b-4e05-80d3-4302df56fad7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.156021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:32.174886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.175435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.175728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.175999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.176672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:32.176967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.178340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6ed3e-e9c5-4446-9cd4-dbf6d0ab8479 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.197186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 83/173] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:32.202316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.202689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.202966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.224338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:32.225549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:32.305182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ed03b-77f1-4655-bb0c-4fbf84333280 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.334209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 84/174] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:32.340528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:32.432377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.433363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.433870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.434386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.441004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.442871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.444752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 91/175] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:32 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:32.452474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:32.469636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:32.483146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9c692-bb9f-4ec7-b946-3aae67dbccd5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.502058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:32.527210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.527877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.528157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.528482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] PATH_INFO: `/v3/users/41b5e9ce17744df388fc2899d20c97a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.529997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] RBAC: Authorizing `identity:get_user(user_id=41b5e9ce17744df388fc2899d20c97a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.531594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.533229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 85/176] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:32 2022] GET /identity/v3/users/41b5e9ce17744df388fc2899d20c97a1 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:32.538983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:32.653707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e0b8-982d-446b-b12d-7ffe382c8d1c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.697543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.698086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.698364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.698634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.699699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c7eda-1792-4235-a39d-bbf219e04b39 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:32.761329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:32.779750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:32.780547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.780923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.781303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.786365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.788038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f200c-16e0-43bd-9416-228707cc0855 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.903513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:32.905419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88851804-b0e9-490f-8a5c-a89084df570e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:32.916316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:32.938814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 86/177] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:32.941852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013d74a-6edd-4d4e-8b49-d3c58a8875af None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:32.942243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013d74a-6edd-4d4e-8b49-d3c58a8875af None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:32.942626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013d74a-6edd-4d4e-8b49-d3c58a8875af None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:32.944483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 87/178] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:32.948752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013d74a-6edd-4d4e-8b49-d3c58a8875af None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.017671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 92/179] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:33.020371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151d915-a8e2-4310-a2c8-c76133eb07ef None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.020653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151d915-a8e2-4310-a2c8-c76133eb07ef None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.020925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151d915-a8e2-4310-a2c8-c76133eb07ef None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.026215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 93/180] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:33.026215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151d915-a8e2-4310-a2c8-c76133eb07ef None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.051649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.074237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013d74a-6edd-4d4e-8b49-d3c58a8875af None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.110711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.111338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.111713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.111992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.123265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d9f99233-8323-414b-a31c-28b6478f159e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:33.128409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:33.129343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:33.130142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 88/181] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:33.130824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd764-c312-4ba9-9ded-75a72d2eee14 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.131738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd764-c312-4ba9-9ded-75a72d2eee14 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.131738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd764-c312-4ba9-9ded-75a72d2eee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.140977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151d915-a8e2-4310-a2c8-c76133eb07ef None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.150334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a97dd764-c312-4ba9-9ded-75a72d2eee14 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:33.151518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a97dd764-c312-4ba9-9ded-75a72d2eee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:33.151668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.178692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.179309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.179617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.179890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] PATH_INFO: `/v3/projects/75c5a82f81e94f918b6738ed3cb6a293` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.181105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] RBAC: Authorizing `identity:get_project(project_id=75c5a82f81e94f918b6738ed3cb6a293)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.182848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.184650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 94/182] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/projects/75c5a82f81e94f918b6738ed3cb6a293 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.185242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.229582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 89/183] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:33.231784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.232470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.233152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.255124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:33.256869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:33.307640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882d897-fbf1-4662-b5f9-af5058202b33 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.339558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 90/184] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:33.339558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.345097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.369323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.370322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.370808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.371291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.374283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.376201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.378935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 95/185] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.384468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.458260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede583ba-671c-41a2-829f-b3fcd6d984f5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.464291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.504932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.504932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.504932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.504932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.505320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:33.505320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.505320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.513055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 91/186] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.513952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.543226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ef3fc-2632-4c77-add0-a5d3d2701a78 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.579168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.579788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.580066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.580392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.581126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=75c5a82f81e94f918b6738ed3cb6a293, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:33.581426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.582859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.583737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.588995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 96/187] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=75c5a82f81e94f918b6738ed3cb6a293&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.588995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.640183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0159bbd-8ff6-4b22-8775-5feff6f22404 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.687507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.688219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.688499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.688768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.689504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=41b5e9ce17744df388fc2899d20c97a1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:33.690354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.692553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa59a25-869b-4805-83fe-5c48ff997a2f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.701629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 92/188] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/role_assignments?user.id=41b5e9ce17744df388fc2899d20c97a1&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.703263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1521-1a9e-4da4-8460-512c6195078e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.703557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1521-1a9e-4da4-8460-512c6195078e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.703838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1521-1a9e-4da4-8460-512c6195078e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.705434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 93/189] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:33.706992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1521-1a9e-4da4-8460-512c6195078e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.720680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecf5aa-7a12-4dbe-a285-c65a4f6943dc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.734967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.764418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.764983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.765267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.765545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.781312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:33.787514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:33.788721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:33.789669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:33.790282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 97/190] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:33.790843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:33.866703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1521-1a9e-4da4-8460-512c6195078e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.901355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:33.913126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.914084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.914566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.914841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.932098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.933594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a69a90-7260-47c3-aa8e-31c38f137d3c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.935445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 94/191] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.937132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.937563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.937835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.948833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6e8ae-5253-4c34-adbf-e34c8e7300bd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:33.957822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:33.959144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:33.988521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:33.989163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:33.990070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:33.990070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:33.991103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:33.992572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:33.994130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 98/192] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:33.994878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.030634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 95/193] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:34.031869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.042755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.110552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35975e3-f029-4e4f-8576-ead244370cfa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.146651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.146938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f493831-dc7d-4d5d-9c27-54c39078215c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.147395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.147592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.147825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.148491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:34.148796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.150189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.160044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 99/194] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:34.160744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.170499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.192442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.193053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.193331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.193598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.265364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.266879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.267288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.272799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691f81-94b9-4293-b8c5-ef5755b82bf3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.309767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.310295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.310575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.310849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.311496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.312987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.340909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 100/195] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:34 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:34.340909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.383928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 96/196] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:34.384805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.407130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.461672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f627b6e-4091-4581-9be5-aeec495926df None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.502043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.503233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.503609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.503881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.504051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fdb7-46ac-493e-9a3d-93eb5127e95b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.505088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.507181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.508345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 101/197] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:34.508937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.545779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.546783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.547588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.547778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.548082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.564124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:34.570929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:34.571867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:34.572661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 97/198] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:34.573309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.642289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66bc65-8a8c-454b-955e-10c413a17635 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.681444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.681887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.682104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.682379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.682649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.683863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.685712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.686995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 102/199] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:34.690770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.709132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d248b-be6b-410c-9381-84b01d6cd7d8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.753271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.753890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.754177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.754431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.820609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.831871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.833663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e725e9c-5c19-44b1-94ca-ad9ed5ec5e72 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:34.833891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.879060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:34.879708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:34.879987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:34.880256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:34.880928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:34.881231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:34.883588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:34.900104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 103/200] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:34.901476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:34.928115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:34.959438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 98/201] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:34.959438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:35.020316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8ef90-b3ce-4b28-a172-8d23389eddc0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.056219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.057637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.058113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.058575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.059512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=5e394b6a9fe6401f81618fd2d4e16f71) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:35.060042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.061706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.068390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 104/202] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/role_assignments?user.id=5e394b6a9fe6401f81618fd2d4e16f71&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:35.069480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:35.086666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66d54d-2c5e-4c9f-a603-0e5fc244420e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.098801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:35.172216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.172870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.173141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.173413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] PATH_INFO: `/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.174749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] RBAC: Authorizing `identity:get_user(user_id=3b09f3dd5e2148f59d5c7bab89e1b4dd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.176393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.178103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 99/203] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:34 2022] GET /identity/v3/users/3b09f3dd5e2148f59d5c7bab89e1b4dd => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:35.183901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:35.244435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adc17a-50f5-4d18-b242-d337127fe07c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.255130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:35.304820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.305470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.305747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.306022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.307156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.308652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.310457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 105/204] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:35 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:35.318830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:35.358589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dfa59-01f9-46c4-aa47-11f45bbd6b4d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.417195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:35.436818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a216b0d-3ca4-4a97-85ab-19bdbc594fa5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.447751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.448481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.448768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.449033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.498571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.499179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.499456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.499751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.500476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=5e394b6a9fe6401f81618fd2d4e16f71) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:35.501458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.505069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.521488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 106/205] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:35 2022] GET /identity/v3/role_assignments?user.id=5e394b6a9fe6401f81618fd2d4e16f71&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:35.564399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.566615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.635086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:35.744584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 100/206] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:35.750410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:35.890173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41443b-e064-4643-a172-2a02d7bacc59 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:35.936924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:35.937559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:35.937899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:35.938178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:35.939351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:35.941065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4875-9fe2-461d-9a66-b5641594f95a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:35.942731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 107/207] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:35 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:35.948809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:36.057117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9552e4-99b5-4042-b53c-e3df81ddf6b7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:36.101920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:36.102529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:36.102804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:36.103073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:36.103897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b, user.id=3b09f3dd5e2148f59d5c7bab89e1b4dd) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:36.104203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:36.105631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8eb5-8c8d-4fe7-ba75-4cbcf827f299 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:36.112544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 101/208] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:35 2022] GET /identity/v3/role_assignments?user.id=3b09f3dd5e2148f59d5c7bab89e1b4dd&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=aee3894c6efa40af958645465c848a05 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:36.972444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6fa7-aa46-4d2f-a78b-905d61413dd7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:36.972993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6fa7-aa46-4d2f-a78b-905d61413dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:36.972993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6fa7-aa46-4d2f-a78b-905d61413dd7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:36.974664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 108/209] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:36.980144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:36.980562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:36.980656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:36.998713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:37.000037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:37.002499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:37.009067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547203c-2dcd-41e1-b8ad-cf61e78e1ef3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.009347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547203c-2dcd-41e1-b8ad-cf61e78e1ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.009618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547203c-2dcd-41e1-b8ad-cf61e78e1ef3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.010944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 109/210] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:37.015953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.016233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.016508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.037884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:37.039253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:37.051678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:37.091625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 102/211] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:37.119244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 110/212] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:37.406619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:37.435587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:37.575026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32792d06-6c1e-4bee-8c22-61941b20300a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:37.621082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:37.623335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.623923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.625158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.628586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8f7f-27cd-42f7-a960-487ad7279059 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:37.638800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:37.649073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ff850610-6d15-4073-9bb0-d3e5f3392ecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:37.657933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:37.659924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:37.660734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 103/213] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:37.665949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58dcf-6953-472f-8173-2c55a07904ed None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.666241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58dcf-6953-472f-8173-2c55a07904ed None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.666524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58dcf-6953-472f-8173-2c55a07904ed None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.668371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 104/214] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:37.669014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb58dcf-6953-472f-8173-2c55a07904ed None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:37.696060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:37.696707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.696986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.697822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.699096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:37.700573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ca998-4d6b-466a-9750-eee1bb9e500f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:37.702304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 111/215] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:37.707291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.707291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.707291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.732718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:37.734012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:37.751313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:37.793098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb58dcf-6953-472f-8173-2c55a07904ed None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:37.827985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:37.828494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 112/216] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:37.828601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.828859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:37.829135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:37.829881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:37.830810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:37.831222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:37.832238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:37.841065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 105/217] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:37.847398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:37.912902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:37.958189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d357bb-7042-455d-8c03-1885bf92c4c7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:37.978460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e595e-14cf-4ca0-8f06-0d6fea847ebf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:37.998888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:37.999663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:37.999960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.000269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] PATH_INFO: `/v3/users/519da4b8f9644c089786bbd8e02923d1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.016040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] RBAC: Authorizing `identity:get_user(user_id=519da4b8f9644c089786bbd8e02923d1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.017416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.019131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 106/218] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:37 2022] GET /identity/v3/users/519da4b8f9644c089786bbd8e02923d1 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:38.024712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.030707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.031646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.032184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.032645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.034703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.037010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07295a49-5f16-464c-b471-71156f550b53 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.060914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.076358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 113/219] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:37 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:38.094394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec676-1e23-40de-9694-4107adea87a8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.094710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec676-1e23-40de-9694-4107adea87a8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.094955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec676-1e23-40de-9694-4107adea87a8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.097024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 114/220] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:38.103163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677239c-de88-4856-af23-79be05438b62 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.103498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677239c-de88-4856-af23-79be05438b62 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.103758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677239c-de88-4856-af23-79be05438b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.125069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b677239c-de88-4856-af23-79be05438b62 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:38.126510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b677239c-de88-4856-af23-79be05438b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:38.152860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50d61-5e2b-4fc8-9182-f5c520e7bca8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.178840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.200417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.201505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.201985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.202425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.206106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 115/221] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:38.206815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677239c-de88-4856-af23-79be05438b62 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.297452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.299975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.328279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677239c-de88-4856-af23-79be05438b62 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.368733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.377504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.378819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.379303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.379826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.391254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:38.398622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:38.399506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:38.400357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 116/222] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:38.407909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.427370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 107/223] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:38.434593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.458713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.545215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc1853-7104-4d36-9c44-a6b5fb460a7d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.565340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9cfdf-64f8-4d9a-8519-cc63086d714b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.583678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.584554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.585071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.585480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.586481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:38.586919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.588863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.598116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 117/224] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:38.598782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.604348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.605197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.605674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.606347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.612448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.614884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.618251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 108/225] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:38.619234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.673617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.725845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9c79-f608-48ef-9457-b9e08f8b7b78 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.729943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.747130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9fadd-1548-4b6d-8725-901c3de3b5bf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.763665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.764438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.764725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.764998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.775670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:38.781330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:38.782232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 118/226] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:38.784543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.790631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.791570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.792056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.792541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.794008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.795687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.797586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 109/227] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:38.799744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.893282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e26f9-b174-4db4-b6b9-63c11b7b24c6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.917453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e8a8-adf0-4a66-b4d6-8daf3ace07f4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:38.919569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:38.931601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.932515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.932744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.933027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.933865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=519da4b8f9644c089786bbd8e02923d1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:38.934073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.935583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:38.942183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 119/228] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/role_assignments?user.id=519da4b8f9644c089786bbd8e02923d1&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:38.943001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:38.980300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:38.980880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:38.981159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:38.981431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:38.982141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:38.982445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:38.983905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.007454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 110/229] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:39.012486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:39.050676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:39.156384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bf8e7-5ed1-43eb-9346-9b6d28092d30 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:39.196740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4b462-55b8-40d6-9068-e512b2858b95 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:39.204052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:39.204798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:39.205249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:39.205561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:39.206850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:39.208728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.212160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 120/230] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:38 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:39.218874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:39.247145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:39.247817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:39.248950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:39.252097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:39.254913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:39.256403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.258037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 111/231] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:39 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:39.263477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:39.301109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:39.418694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99f54e-383b-42fe-a90b-ef90b3a9a1cb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:39.448344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:39.467172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a8e01-7d30-4cd3-9464-00ef61e83d9d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:39.502587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:39.503318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:39.503632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:39.503915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/5e394b6a9fe6401f81618fd2d4e16f71/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:39.506612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, user_id=5e394b6a9fe6401f81618fd2d4e16f71, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:39.508104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c1e05-43aa-459a-9f7b-015e6087018b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.512834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:39.513483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:39.513764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:39.514037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:39.515839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:39.517279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.521250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 112/232] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:39 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:39.525532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:39.544477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 121/233] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:39 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/5e394b6a9fe6401f81618fd2d4e16f71/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:11:39.650719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4199f-fc74-4142-804d-b845410e3c28 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:39.688565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:39.689765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:39.690094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:39.690458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/5e394b6a9fe6401f81618fd2d4e16f71/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:39.692869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, user_id=5e394b6a9fe6401f81618fd2d4e16f71, role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:39.694434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aea1c4-73f9-44c3-9797-dd994ed5ddeb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:39.723619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 113/234] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:39 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/5e394b6a9fe6401f81618fd2d4e16f71/roles/da0177264e704fa19ac1889c21e21eb5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:40.737662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982203-bc16-4728-a567-18102dc6845d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:40.738421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982203-bc16-4728-a567-18102dc6845d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:40.738421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982203-bc16-4728-a567-18102dc6845d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:40.739577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 122/235] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:40.758105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485f1c8-d708-44af-8caf-da528ce0f2be None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:40.758737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485f1c8-d708-44af-8caf-da528ce0f2be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:40.758737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485f1c8-d708-44af-8caf-da528ce0f2be None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:40.760338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 114/236] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:40.761548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:40.761812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:40.762095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:40.766684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:40.767251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:40.768052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:40.803060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:40.804347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:40.805786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:40.809276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:40.824006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:40.944250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 115/237] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:40.952351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 123/238] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:41.379726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:41.468833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:41.515909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7af69-faba-404a-a845-8965cadecda2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:41.516857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:41.553945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:41.554235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:41.554524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:41.554803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:41.569656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a3834974-765d-4de4-b8c7-cf3caf1bbaa7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:41.576044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:41.577438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:41.578230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 116/239] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:41.584712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28faff-585b-4ff3-90c6-e0ad52c591e5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:41.584712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28faff-585b-4ff3-90c6-e0ad52c591e5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:41.584712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28faff-585b-4ff3-90c6-e0ad52c591e5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:41.584712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 117/240] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:41.586845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28faff-585b-4ff3-90c6-e0ad52c591e5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:41.592003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:41.604276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814de8-5aaf-4098-80a1-a3e83db7459d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:41.644976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:41.645930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:41.646201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:41.646670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:41.647942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:41.649515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996d6e-b566-4523-825d-59d099463346 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:41.656237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 124/241] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:41.657763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:41.658222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:41.658791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:41.677910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:41.679426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:41.729757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28faff-585b-4ff3-90c6-e0ad52c591e5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:41.755247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 125/242] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:41.756629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:41.788690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:41.789900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:41.790318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:41.790725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:41.791549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:41.791989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:41.794501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:41.806867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 118/243] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:41.815200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:41.821097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:41.931133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:41.944668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703e49e-dc0c-45a4-ad88-814489610a46 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:41.997083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f8009-996e-4460-8210-da905b714d59 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.005499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.006193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.006669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.007130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.008145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.009922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.043801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.049935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.050440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.051565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] PATH_INFO: `/v3/users/f140dd6d20024e5a87e4130a7b79aa90` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.054768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 126/244] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:41 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:42.078566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] RBAC: Authorizing `identity:get_user(user_id=f140dd6d20024e5a87e4130a7b79aa90)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.080689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.082786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 119/245] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:41 2022] GET /identity/v3/users/f140dd6d20024e5a87e4130a7b79aa90 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:42.088505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.099272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.257533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.294246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901077-7d65-4232-aabc-4269464cbf0e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.298075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a596491-4aff-410a-8bdd-5aa19a6cf3a1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.329862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.343705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.344622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.344750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.344954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.345176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.345271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.345555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.345995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.364246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3ac74b4c-57f4-489d-82d4-0278619f07a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:42.369421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:42.370364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:42.371479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 120/246] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:42.372473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7d25-c8c0-4aec-a09f-a85a48d3b236 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.372473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7d25-c8c0-4aec-a09f-a85a48d3b236 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.372473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7d25-c8c0-4aec-a09f-a85a48d3b236 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.375029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 121/247] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:42.378266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc7d25-c8c0-4aec-a09f-a85a48d3b236 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.441575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.443659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f6a5d-972d-45f9-90df-95d7f178c29a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.517601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc7d25-c8c0-4aec-a09f-a85a48d3b236 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.558268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.563307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.563989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.564293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.564735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.565683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:42.565962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.567342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0b79f-4b60-4a37-8099-879afdaa624f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.575939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 127/248] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:42.577030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.577357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.577868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.580377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 122/249] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:42.582103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b604c0-8ef0-4f0d-bed1-7ae392dec4be None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.582538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b604c0-8ef0-4f0d-bed1-7ae392dec4be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.583054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b604c0-8ef0-4f0d-bed1-7ae392dec4be None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.584585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 123/250] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:42.588422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b604c0-8ef0-4f0d-bed1-7ae392dec4be None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.598151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:42.599359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:42.654951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.680893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 128/251] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:42.681759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.724257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b604c0-8ef0-4f0d-bed1-7ae392dec4be None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.764082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:42.764819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.765093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.765365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.766621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:42.768852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe53118-6485-412e-a7ad-b052ca7b0145 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:42.771157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 124/252] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:42.772307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:42.772580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:42.772848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:42.800415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:42.801619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:42.882097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 125/253] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:42.882962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:42.928596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6e414-69ae-4b43-ab7a-ed49fe13dcaa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:42.936134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:42.970595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:43.004149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768125f0-4546-4780-9f79-fb4bf7be9320 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.004484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.004869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.005149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.005525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] PATH_INFO: `/v3/users/519da4b8f9644c089786bbd8e02923d1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.006810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] RBAC: Authorizing `identity:get_user(user_id=519da4b8f9644c089786bbd8e02923d1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:43.008301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:43.010201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 129/254] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/users/519da4b8f9644c089786bbd8e02923d1 => generated 297 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:43.025263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.046905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.047515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.047863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.048133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.049882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=f140dd6d20024e5a87e4130a7b79aa90) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:43.050184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:43.051719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:43.057929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 126/255] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:42 2022] GET /identity/v3/role_assignments?user.id=f140dd6d20024e5a87e4130a7b79aa90&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=aee3894c6efa40af958645465c848a05 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:43.204041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7737-78d9-4b62-ba5f-6fc62f407b99 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.225084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.280878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.281835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.282299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.283273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.284973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:43.286693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:43.289075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 130/256] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:43.312697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.376744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860e0e9-c037-4ba8-b8c6-c85ecf7cdcc8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.404154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:43.413609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.414351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.414859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.415145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.427565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:43.432938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:43.434007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:43.434981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:43.435625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 127/257] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:43.436147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.501080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:43.553622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee1da-658c-4f42-81fe-232e3dbcf59c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.589943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.591029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.591526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.591836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.604504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:43.612343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:43.613826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:43.614682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:43.615318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 128/258] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:43.619841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.650592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:43.677547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698ac19-aa81-4a28-bfac-c828d81b4cbd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.736743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71bbe0-c27a-485b-b27c-631cfaec186e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:43.765894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.766555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.766827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.767116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/519da4b8f9644c089786bbd8e02923d1/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.771560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, user_id=519da4b8f9644c089786bbd8e02923d1, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:43.773033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:43.797909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 131/259] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:43 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/519da4b8f9644c089786bbd8e02923d1/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:43.799345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:43.812844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:43.813735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:43.814854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:43.816066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:43.816957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:43.819136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:43.820903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:43.843504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 129/260] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:43.846880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.024911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cbe24-99cd-4c3d-9e4c-b35049cdc8d1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.047972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.055658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.072963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392736a-ec8f-49be-8d12-6c150afc7709 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.100118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.100748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.101022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.101287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.101974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:44.102275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.103713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.114117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 132/261] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:44.119774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.129528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.130450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.130874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.131297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.132913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.135363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.137581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 130/262] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:43 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:44.144230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.260937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3393fe-76be-4507-b738-aedcf904b4ee None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.270425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd25d4-be45-4ddd-bf03-cf060a51176d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.299455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.300129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.300403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.300862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.311188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.312306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.312782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.313246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] PATH_INFO: `/v3/users/5e394b6a9fe6401f81618fd2d4e16f71` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.315377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] RBAC: Authorizing `identity:get_user(user_id=5e394b6a9fe6401f81618fd2d4e16f71)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.317196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.319081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 133/263] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/users/5e394b6a9fe6401f81618fd2d4e16f71 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:44.325290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.331011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.377744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.378104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.380421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.437647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677358-fe94-45a5-995f-828de62212fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.486382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.487479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.488008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.488478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 131/264] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:44.488687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.494269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.556347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.603643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5c7ed-bbc7-44fb-8389-5c270a807401 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.612571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.614327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.638741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.639389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.639703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.639983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.641405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.643210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.645444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 132/265] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:44.651899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.700212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.765000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104664c8-1d3a-4827-9357-18e883f47ab5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:44.804696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:44.806028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:44.806522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:44.807118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:44.808204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=5e394b6a9fe6401f81618fd2d4e16f71) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:44.808719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:44.810547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:44.814099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 134/266] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:44.817407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 133/267] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/role_assignments?user.id=5e394b6a9fe6401f81618fd2d4e16f71&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:44.819912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:44.824056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:44.975357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b97a-4387-45c9-a44f-2b6c6abf43cc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:45.030379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:45.031166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.031479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.031785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.033603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:45.035096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f295f23-657e-4d85-b9d0-a27bd6543e62 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:45.036897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 135/268] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:44 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:45.043198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:45.145932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49c6e-0bae-4c81-b9c9-9079ca3ae3a5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.146251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49c6e-0bae-4c81-b9c9-9079ca3ae3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.146479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49c6e-0bae-4c81-b9c9-9079ca3ae3a5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.147971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 136/269] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:45.153047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059757-3702-44ab-93ae-4fa6567a5044 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.153326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059757-3702-44ab-93ae-4fa6567a5044 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.153595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa059757-3702-44ab-93ae-4fa6567a5044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.184557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aa059757-3702-44ab-93ae-4fa6567a5044 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:45.185262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d09a-c324-49e5-90a7-310f1f6d1b75 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:45.185762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa059757-3702-44ab-93ae-4fa6567a5044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:45.194623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:45.223078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:45.224944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.226129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.226716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.231984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=5e394b6a9fe6401f81618fd2d4e16f71) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:45.233129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:45.234747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9acb5-40f8-493b-b8aa-c78022c08770 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:45.243499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 134/270] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:45 2022] GET /identity/v3/role_assignments?user.id=5e394b6a9fe6401f81618fd2d4e16f71&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:45.289876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 137/271] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:45.524275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db8f5-28c9-49fe-91f3-f8c6248484f7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.524774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db8f5-28c9-49fe-91f3-f8c6248484f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.525194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db8f5-28c9-49fe-91f3-f8c6248484f7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.526915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 135/272] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:45.533624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.534083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.534505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.553371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:45.554846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:45.624835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 138/273] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:45.715481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671db8f5-28c9-49fe-91f3-f8c6248484f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:45.775224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:45.837528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671db8f5-28c9-49fe-91f3-f8c6248484f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:45.881758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:45.882398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:45.882671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:45.882935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:45.884123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:45.885542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:45.888143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 136/274] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:45.893895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:45.904950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:45.986701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:46.024403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265e1fb-f07f-4029-b3e9-f769085d8cc8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.026593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec398e-3b9f-4fdc-864b-7a89ced61b48 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.046863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:46.069112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.069714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.069990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.070255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.082453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.083597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.084017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.084489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.085036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:46.085518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:46.087185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:46.090763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:46.091687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:46.092490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 137/275] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:46.100540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:46.116338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 139/276] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:11:45 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:46.232188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766d7a-33dd-48d4-8977-284783491312 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.277944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.279182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.279696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.280162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.281078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:46.281663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:46.283964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e412c-c5a4-40a3-8762-c0c73355eaf2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:46.293959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 138/277] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:46.301713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:46.331151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7c61-3deb-424a-a0ad-87dce985e0f1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.331924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7c61-3deb-424a-a0ad-87dce985e0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.332198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a7c61-3deb-424a-a0ad-87dce985e0f1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.333780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 139/278] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:46.338931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.339211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.339480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.358702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:46.359922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:46.398247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:46.434324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b470e-7410-4950-8954-dbd58fb2095d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.442824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 140/279] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:46.470872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:46.480727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.481547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.482025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.482439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] PATH_INFO: `/v3/users/5e4168226c1c483a982c6ad0948c4dc7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.497093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] RBAC: Authorizing `identity:get_user(user_id=5e4168226c1c483a982c6ad0948c4dc7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:46.498802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:46.500936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 140/280] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:46 2022] GET /identity/v3/users/5e4168226c1c483a982c6ad0948c4dc7 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:46.507271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:46.613195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a654e-0fe8-454c-8ae3-140a506a05b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.660855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.661778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.662230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.662678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.663616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, user.id=5e4168226c1c483a982c6ad0948c4dc7, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:46.664095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:46.665694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4977a54-43b9-476d-a0e0-ed95d5011d7c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:46.671348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 141/281] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 21:11:46 2022] GET /identity/v3/role_assignments?user.id=5e4168226c1c483a982c6ad0948c4dc7&scope.system=all&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:46.726637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:46.836257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ef200-61db-431c-9b18-a7d98ebb1961 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:46.841892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a334-ddd4-4040-bdcd-50cf8f15b017 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.842819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a334-ddd4-4040-bdcd-50cf8f15b017 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.843288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a334-ddd4-4040-bdcd-50cf8f15b017 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.845718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 142/282] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:46.852249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.852768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.853119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.876829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:46.877899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:46.878418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:46.878868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:46.889849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:46.893088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:46.901888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:46.912039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:46.913013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:46.913856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:46.917912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 141/283] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:46.927330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:46.964348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:47.012269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:47.034422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 143/284] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:47.108704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f639f-39a2-4a05-91e5-1101bb84c081 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.162111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.162737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.163026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.163287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.163994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:47.164297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:47.165684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:47.178982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 142/285] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:47.184569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:47.347483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce45c0-6ef5-4a51-ad12-0c284d0e9690 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.377801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:47.400271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.400916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.401191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.401457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] PATH_INFO: `/v3/users/f140dd6d20024e5a87e4130a7b79aa90` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.402648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] RBAC: Authorizing `identity:get_user(user_id=f140dd6d20024e5a87e4130a7b79aa90)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:47.404380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:47.410944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 144/286] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:47 2022] GET /identity/v3/users/f140dd6d20024e5a87e4130a7b79aa90 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:47.412408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:47.487889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:47.553550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09271a5-25fb-436b-939a-1b32c8a82f1f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.571922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:47.613805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940822d1-be3f-49fe-8523-2d8dd1887604 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.620938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.621570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.621843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.622112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.644336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:47.649849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:47.650723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:47.651551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 143/287] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:47.670548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:47.678276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.679195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.679760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.680262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.681635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:47.683257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:47.685468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 145/288] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:47 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:47.691300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:47.795257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7d89c-7396-4e28-b158-b8d838573ee1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.797224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:47.831879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.832927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.833443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.833918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/f140dd6d20024e5a87e4130a7b79aa90/roles/aee3894c6efa40af958645465c848a05` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.836083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, user_id=f140dd6d20024e5a87e4130a7b79aa90, role_id=aee3894c6efa40af958645465c848a05)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:47.837724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:47.854144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 146/289] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:11:47 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/users/f140dd6d20024e5a87e4130a7b79aa90/roles/aee3894c6efa40af958645465c848a05 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:47.877969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d985a-b5b0-4e18-bedb-3fe4317096c5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:47.942668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:47.943219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:47.943495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:47.943795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:47.944472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:47.944774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:47.946168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:47.964391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 144/290] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:47.971480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:48.117709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0324b4c-2e4a-443f-8ec3-9627239d358b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:48.154229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:48.155754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:48.156312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:48.156794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] PATH_INFO: `/v3/users/519da4b8f9644c089786bbd8e02923d1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:48.158771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] RBAC: Authorizing `identity:get_user(user_id=519da4b8f9644c089786bbd8e02923d1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:48.160425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:48.162215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 147/291] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:47 2022] GET /identity/v3/users/519da4b8f9644c089786bbd8e02923d1 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:48.168208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:48.201261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:48.296189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dc39d-33e6-4876-8ad7-e447ed2a28bf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:48.348269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:48.348905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:48.349186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:48.349457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:48.441556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:48.448865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:48.487553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:48.588101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 145/292] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:48.593564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:48.631567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:48.705435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe9bd4-5ed2-4337-95ed-f9bf4f9653b6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:48.740548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ef99-525c-401f-891c-58d489156907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:48.741102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ef99-525c-401f-891c-58d489156907 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:48.741382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ef99-525c-401f-891c-58d489156907 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:48.741660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ef99-525c-401f-891c-58d489156907 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:48.742780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513ef99-525c-401f-891c-58d489156907 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:48.744291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513ef99-525c-401f-891c-58d489156907 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:48.746340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 148/293] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:48 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:48.752696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:48.894949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513ab17-a921-4490-85bb-76ef87e18b24 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:48.937089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:48.943839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:48.944646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:48.944927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:48.945199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:48.945944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=519da4b8f9644c089786bbd8e02923d1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:48.946248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:48.947905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0189e-1f00-42a5-b876-d09ac3d374a8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:48.955155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 146/294] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:48 2022] GET /identity/v3/role_assignments?user.id=519da4b8f9644c089786bbd8e02923d1&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:49.024840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54567-88eb-4506-999d-5cf65ffb9ed4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.025342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54567-88eb-4506-999d-5cf65ffb9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.025342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54567-88eb-4506-999d-5cf65ffb9ed4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.026636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 149/295] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:49.032871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14bdc-af73-4f1d-a935-e6f72521afb6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.033191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14bdc-af73-4f1d-a935-e6f72521afb6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.033431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14bdc-af73-4f1d-a935-e6f72521afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.059343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ad14bdc-af73-4f1d-a935-e6f72521afb6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:49.060665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ad14bdc-af73-4f1d-a935-e6f72521afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:49.158254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 147/296] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:49.235389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b5b8-e0ee-4afc-a48d-cd4c11cf0259 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.236363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b5b8-e0ee-4afc-a48d-cd4c11cf0259 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.236840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b5b8-e0ee-4afc-a48d-cd4c11cf0259 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.238340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 150/297] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:49.243781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.244242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.244318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.263094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:49.264351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:49.335045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 148/298] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:49.508078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b5b8-e0ee-4afc-a48d-cd4c11cf0259 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:49.616600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:49.621062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b5b8-e0ee-4afc-a48d-cd4c11cf0259 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:49.633597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:49.659608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:49.660238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.660514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.660780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.671095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4a0c80a5-a189-43f3-92aa-9604568530c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:49.676051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:49.676937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 151/299] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:49.681015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306492-91cd-4a38-b48a-c5619e4fe94f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.682169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306492-91cd-4a38-b48a-c5619e4fe94f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.682169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306492-91cd-4a38-b48a-c5619e4fe94f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.682169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 152/300] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:49.683643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306492-91cd-4a38-b48a-c5619e4fe94f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:49.700121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:49.748158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8caee8-477a-4243-8582-24345ce0d6ab None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:49.784986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:49.785621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.785900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.786176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.787288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:49.788840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb56002-4072-4179-b615-2c1fd1694a97 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:49.790410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 149/301] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:49.792702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.792981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.793252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.798455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306492-91cd-4a38-b48a-c5619e4fe94f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:49.812837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:49.814059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:49.836348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:49.837283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:49.837555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:49.837821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:49.838481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:49.838775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:49.840155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:49.849251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 153/302] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:49.850138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:49.875594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:49.903558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 150/303] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:49.903558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:49.964808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:49.998340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8954eb-2aa0-48b3-8eb2-d5628d686db5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.023049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f014a-fe94-4d35-8ce7-503a200f8025 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.044486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.046291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.046972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.047386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] PATH_INFO: `/v3/groups` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.048501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.050768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.060183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.060952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.061231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.061753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] PATH_INFO: `/v3/users/bb400f05858a49a28f9d6eeaf3518e36` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.072350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 154/304] 10.222.0.46 () {60 vars in 1134 bytes} [Fri Sep 9 21:11:49 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:11:50.077861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] RBAC: Authorizing `identity:get_user(user_id=bb400f05858a49a28f9d6eeaf3518e36)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.080003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.081792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 151/305] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:49 2022] GET /identity/v3/users/bb400f05858a49a28f9d6eeaf3518e36 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:50.088180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.184453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.195392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:50.230698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3d373-3eb0-46fb-97da-3bb76707540c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.284468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.285486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.285997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.286452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.287366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, user.id=bb400f05858a49a28f9d6eeaf3518e36, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:50.287880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.289483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe875b-6819-45ae-8d77-e5e9f28142c0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.295446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 155/306] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/role_assignments?user.id=bb400f05858a49a28f9d6eeaf3518e36&scope.system=all&role.id=aee3894c6efa40af958645465c848a05 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:50.310984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee045d2-ef30-458f-aba1-6d5cd3727e0d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.324991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:50.357808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.358471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.358750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.359022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.372745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-20b13b1d-6f13-4a2a-9cd5-0852dbba4dd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:50.378799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:50.379864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:50.380948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 152/307] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:50.381762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56a24-a86d-40f2-bdf8-40105e2c7542 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.381762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56a24-a86d-40f2-bdf8-40105e2c7542 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.381762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56a24-a86d-40f2-bdf8-40105e2c7542 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.382369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 156/308] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:50.387976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.388253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.388519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.389118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc56a24-a86d-40f2-bdf8-40105e2c7542 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.408383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:50.409658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:50.491658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 153/309] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:50.518244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc56a24-a86d-40f2-bdf8-40105e2c7542 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.554665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.555600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.556087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.556536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.557426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:50.557906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.559441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.573518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 157/310] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:50.575078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.680443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:50.683980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ff68-1046-4a7f-bd07-3a3e0f993efd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.723667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.724307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.724589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.724864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] PATH_INFO: `/v3/users/5e4168226c1c483a982c6ad0948c4dc7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.726070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] RBAC: Authorizing `identity:get_user(user_id=5e4168226c1c483a982c6ad0948c4dc7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.727514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.729115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 154/311] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/users/5e4168226c1c483a982c6ad0948c4dc7 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:50.735058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.748202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:50.775622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:50.858352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d770b-b696-410f-a961-3baa69e249f9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.901265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a85b3-49c9-410e-9c12-980d50819794 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:50.909168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.910681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.911144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.911625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] PATH_INFO: `/v3/system/users/5e4168226c1c483a982c6ad0948c4dc7/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.914190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e4168226c1c483a982c6ad0948c4dc7, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:50.916105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:50.929973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 158/312] 10.222.0.46 () {58 vars in 1334 bytes} [Fri Sep 9 21:11:50 2022] PUT /identity/v3/system/users/5e4168226c1c483a982c6ad0948c4dc7/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:11:50.956849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:50.957998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:50.958556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:50.959100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:50.973453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:50.979247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:50.980224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:50.981052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:50.983919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 155/313] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:50.995941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:51.076379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:51.114129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9fa6-df4f-44f4-8ab6-951277ea5d4c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:51.153179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:51.154101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:51.154373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:51.154635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:51.155300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:51.155636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:51.157117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:51.166674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 159/314] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:51.172275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:51.203242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:51.283164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae873a-e033-445a-b8b5-bdd55d7b7c37 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:51.318563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:51.319194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:51.319462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:51.319765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] PATH_INFO: `/v3/users/f140dd6d20024e5a87e4130a7b79aa90` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:51.320902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] RBAC: Authorizing `identity:get_user(user_id=f140dd6d20024e5a87e4130a7b79aa90)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:51.322340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:51.324279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 156/315] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:51 2022] GET /identity/v3/users/f140dd6d20024e5a87e4130a7b79aa90 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:51.332910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:51.436707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da787-11f0-4c0a-8c5f-cbd8f643e503 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:51.469060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:51.475371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:51.476027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:51.476312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:51.476587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:51.548419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:51.549888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:51.659153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 160/316] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:51.668604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:51.774913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:51.782640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274630dd-35df-4b1b-9fd5-35eea1756f2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:51.818471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:51.819438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:51.819951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:51.820449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:51.821841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:51.823507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011699e1-32a3-4dcc-8a65-0a546e29523b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:51.825753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 157/317] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:51 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:51.832054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:51.863330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:51.943200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfae2a-0c0b-4a8e-8d87-22f941434225 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:51.980185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:51.980797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:51.981075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:51.981350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:51.982160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, scope.project.id=62067e6078ac4fe69c558de35c40ee8e, user.id=f140dd6d20024e5a87e4130a7b79aa90) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:51.982471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:51.983919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac48a-f70c-4318-a5fb-d60b07ff2d65 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:51.990715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 161/318] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:11:51 2022] GET /identity/v3/role_assignments?user.id=f140dd6d20024e5a87e4130a7b79aa90&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=aee3894c6efa40af958645465c848a05 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:52.503023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314681-9831-4402-9c10-49f8b699bd47 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:52.503749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314681-9831-4402-9c10-49f8b699bd47 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:52.503749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3314681-9831-4402-9c10-49f8b699bd47 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:52.505066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 158/319] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:52.510048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:52.510420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:52.510592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:52.529421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:52.530617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:52.571340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68a8b-ae05-4766-a623-4539e4daaca0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:52.571658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68a8b-ae05-4766-a623-4539e4daaca0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:52.571921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68a8b-ae05-4766-a623-4539e4daaca0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:52.573390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 159/320] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:52.578003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:52.578287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:52.578561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:52.596294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:52.597484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:52.601239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 162/321] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:52.675277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 160/322] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:52.871774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:52.900378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:52.943706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:52.981341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f68b3-cbb6-414b-af85-d969bd8a9b34 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.018796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.019435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.019749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.020022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.021304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.022541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.024141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 163/323] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:11:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:53.029784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.042619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:53.052095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e48f7-fc17-461c-bd7f-6d297401278f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.092629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.093284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.093626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.093833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.106996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:53.113898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:53.114864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:53.115716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:53.116363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 161/324] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:53.122200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.137878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:53.145976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9591-2ac3-4b25-a6b7-f56b8b9d0017 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.182559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.183482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.183797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.184071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] PATH_INFO: `/v3/groups` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.184708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.186586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9edd5-090c-4385-aac1-162ba408d1d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.211569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 164/325] 10.222.0.46 () {60 vars in 1134 bytes} [Fri Sep 9 21:11:53 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:11:53.218081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca031270-e5eb-479d-b7e1-ff53632a81b9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.218413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca031270-e5eb-479d-b7e1-ff53632a81b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.218619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca031270-e5eb-479d-b7e1-ff53632a81b9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.220946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 165/326] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:11:53.226848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.227176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.227395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.246784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:53.248054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:53.250653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ad859-5374-46e3-9117-e93b03c3f160 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.297617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.298912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.299194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.299289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.299995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:53.300302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.301734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.311025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 162/327] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:53.318253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.329714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 166/328] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:53.342194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:53.435344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c5a53-35a7-4e56-b1d2-868ab9880721 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.469234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.469924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.470204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.470478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] PATH_INFO: `/v3/users/bb400f05858a49a28f9d6eeaf3518e36` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.471642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] RBAC: Authorizing `identity:get_user(user_id=bb400f05858a49a28f9d6eeaf3518e36)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.473099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.474958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 163/329] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:53 2022] GET /identity/v3/users/bb400f05858a49a28f9d6eeaf3518e36 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:53.480689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.572653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:53.592247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1c09d-b02d-4c92-b515-bd5642c9c9c4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.600216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.639222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.640617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.640617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.640617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] PATH_INFO: `/v3/system/users/bb400f05858a49a28f9d6eeaf3518e36/roles/aee3894c6efa40af958645465c848a05` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.644185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bb400f05858a49a28f9d6eeaf3518e36, role_id=aee3894c6efa40af958645465c848a05)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.645735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.660631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 167/330] 10.222.0.46 () {58 vars in 1334 bytes} [Fri Sep 9 21:11:53 2022] PUT /identity/v3/system/users/bb400f05858a49a28f9d6eeaf3518e36/roles/aee3894c6efa40af958645465c848a05 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:11:53.729079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:53.748662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e70e4-6be3-46a5-ad14-4d547b0e32bf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.789480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7935-586d-48c1-ae34-75408583219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.790133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7935-586d-48c1-ae34-75408583219c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.790411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7935-586d-48c1-ae34-75408583219c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.790682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b7935-586d-48c1-ae34-75408583219c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.803753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b7935-586d-48c1-ae34-75408583219c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-464b7935-586d-48c1-ae34-75408583219c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:53.810126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:53.811098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:53.811969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 164/331] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:11:53.818008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:53.934490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42155cf0-16a9-49d9-bc20-92537baec276 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:53.974752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:53.975365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:53.975674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:53.975941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:53.976608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:53.976902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:53.978257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:53.987729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 168/332] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:53.994744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7935-586d-48c1-ae34-75408583219c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:54.127718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b7935-586d-48c1-ae34-75408583219c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:54.167714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:54.168355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:54.168699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:54.168979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] PATH_INFO: `/v3/users/5e4168226c1c483a982c6ad0948c4dc7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:54.170140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] RBAC: Authorizing `identity:get_user(user_id=5e4168226c1c483a982c6ad0948c4dc7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:54.171642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971251f-6347-4118-86b4-8046bf998a27 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:54.174129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 165/333] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:53 2022] GET /identity/v3/users/5e4168226c1c483a982c6ad0948c4dc7 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:54.180076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:54.196582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:54.309311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2634b1-9c32-45b4-a27c-f329d4a03143 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:54.353444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:54.354054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:54.354333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:54.354607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:54.355337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] RBAC: Adding query filter params (role.id=869347abd4b848d1abcc7f9e6ffc94a9, user.id=5e4168226c1c483a982c6ad0948c4dc7, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:54.355675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:54.357107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1c3d1-c34c-4619-afb7-2e0958e21001 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:54.371866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 169/334] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 21:11:54 2022] GET /identity/v3/role_assignments?user.id=5e4168226c1c483a982c6ad0948c4dc7&scope.system=all&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:55.862592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265f71-a676-4537-9c59-f614c0d880e7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:55.863462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265f71-a676-4537-9c59-f614c0d880e7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:55.863914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265f71-a676-4537-9c59-f614c0d880e7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:55.865691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 166/335] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:55.871314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc34b86-00a7-4d31-8c6e-2236a9f7bbc0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:55.871790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc34b86-00a7-4d31-8c6e-2236a9f7bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:55.871903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc34b86-00a7-4d31-8c6e-2236a9f7bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:55.891095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8dc34b86-00a7-4d31-8c6e-2236a9f7bbc0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:55.896492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8dc34b86-00a7-4d31-8c6e-2236a9f7bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:55.973920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:55.980394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 170/336] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:56.285485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25b73-c7cd-4a10-8c4c-9e37a448a8f6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.286189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25b73-c7cd-4a10-8c4c-9e37a448a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.286716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be25b73-c7cd-4a10-8c4c-9e37a448a8f6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.288493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 167/337] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:56.294306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.294588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.294858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.317847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:56.319037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:56.381947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425750d-6173-4fe1-b482-4804462a9f5e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.382620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425750d-6173-4fe1-b482-4804462a9f5e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.383116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425750d-6173-4fe1-b482-4804462a9f5e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.384979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 168/338] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:56.401274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.401889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.402361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.415988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 171/339] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:56.440566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:56.441968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:56.452084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:56.525169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 169/340] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:56.575737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:56.700925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a33dd3-ccbe-49dc-9dbb-1bd90b672026 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:56.730164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:56.743186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:56.743884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.744199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.744468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.746275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:56.762879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:56.764008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:56.764869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 172/341] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:56.784861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:56.897501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640af173-8477-44e2-815c-c14dac3f5bae None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:56.955127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4648e-4935-43bb-8804-ad6824f60c5b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:56.957054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:56.957935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:56.958404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:56.958865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:56.971619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:56.976897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:56.977765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 21:11:56.978565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:11:56.981131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 170/342] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:56.988524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.000177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.010920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.012810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.014720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.015190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.016100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:57.018613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.020288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.042575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 173/343] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:57.050201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.138150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c1c5-3df0-46f1-b374-55abc1f88dee None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.175307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.202590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.203244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.203523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.203951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.204804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:57.205107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.208029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.220858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 171/344] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:57.221757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.284229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b295-a4bf-4487-a144-cd5f3a830134 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.303024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.349506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb39cdf-feba-4a00-8d00-bf0a07733007 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.354644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.355568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.356057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.356633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.369632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:57.376565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:57.377498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:57.378339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:57.381205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 174/345] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:57.382286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.392012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.393209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.393487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.393756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.488197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.489941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.517042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7346-c13d-43c1-b406-9d0fdbd8f0d6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.529232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.557397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.559697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.559697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.559697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] PATH_INFO: `/v3/users/bb400f05858a49a28f9d6eeaf3518e36` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.560241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] RBAC: Authorizing `identity:get_user(user_id=bb400f05858a49a28f9d6eeaf3518e36)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.561861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5541d2c-233d-4bd5-9ce4-c04cc480a761 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.564515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 175/346] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/users/bb400f05858a49a28f9d6eeaf3518e36 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:57.565818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020ed-4e7a-4039-a4e9-5a02e06a319a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.566173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020ed-4e7a-4039-a4e9-5a02e06a319a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.566526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020ed-4e7a-4039-a4e9-5a02e06a319a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.571204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 176/347] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:57.571204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7020ed-4e7a-4039-a4e9-5a02e06a319a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.575859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.619797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 172/348] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:57.620593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.690270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7020ed-4e7a-4039-a4e9-5a02e06a319a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.728430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.729341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.729806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.730255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.731461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:57.731977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.733617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb65a0-c189-4d98-b621-9939379250cb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.741676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884471c-332c-4218-b93d-eb56b370ea89 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:57.743363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 177/349] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:57.745003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.745468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.745917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.765629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:57.767765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:57.787131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:57.787958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:57.788310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:57.788893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:57.789539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] RBAC: Adding query filter params (role.id=aee3894c6efa40af958645465c848a05, user.id=bb400f05858a49a28f9d6eeaf3518e36, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:57.789960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:57.791511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:57.806959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 173/350] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/role_assignments?user.id=bb400f05858a49a28f9d6eeaf3518e36&scope.system=all&role.id=aee3894c6efa40af958645465c848a05 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:57.808054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.809351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.848127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:57.849055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 178/351] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:57.850164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:57.977969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6ce1-7152-4e2a-a656-20cb25f8b98d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.022787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a3064-e7ff-4cd0-9cf7-84a0f09d267f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.041314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.042059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.042439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.042744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.047950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.047950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.047950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 174/352] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:58.053564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.079136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.083514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.084207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.084665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.145500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.175832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.187314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fb3f-aa6b-4d24-a3b2-234a0ab4f891 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.194341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.196292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.224782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.225537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.226131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.226131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.232618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.234057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1e29-fe5e-48d5-885b-5c5548601ea9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.236720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 175/353] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:11:58.237918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e186-1bf5-41e1-a8be-d6e69f768a1d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.238199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e186-1bf5-41e1-a8be-d6e69f768a1d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.238471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e186-1bf5-41e1-a8be-d6e69f768a1d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.243570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 176/354] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:58.245230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.245520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.245798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.271342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:58.272948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:58.368801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 179/355] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:58.369816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.379010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 177/356] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:11:58.379908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.489902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.527166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.547077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16986-2a95-48b8-b135-01e1ddc63a41 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.551697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b59c-0ebd-4332-a730-4071c6f586e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.589743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.590362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.590641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.590917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.591687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:58.592076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.593434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.595256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.595945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.596220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.596491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.597638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.599081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.599844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 180/357] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:58.600831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.600946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 178/358] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:58.606517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.737052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddb0a6-91dc-4f78-b785-32fa9157c659 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.764325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7d93-68fe-4458-bb20-b0307782c742 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.776195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.776499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.776777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.777045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.779525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.791425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d0fd7715-30c2-4286-95f8-dd1f62eabfff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:58.796975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:58.798401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:58.799277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 181/359] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:58.805600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e95b-117f-4f0e-ba32-987b0267692a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.805600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e95b-117f-4f0e-ba32-987b0267692a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.805600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e95b-117f-4f0e-ba32-987b0267692a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.805600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 182/360] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:11:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:11:58.805600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e95b-117f-4f0e-ba32-987b0267692a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.820614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.824180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.824180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.824180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:58.825481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:58.826758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:58.835037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 179/361] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:58.835994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:58.865939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:58.938614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e95b-117f-4f0e-ba32-987b0267692a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.978141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7eeb7-2d16-48d2-bb7c-5cd315fb0574 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:58.986567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:58.987783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:58.988065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:58.988342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.005598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-74206dc8-0031-44af-85cc-e03b2b14dbd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:59.011386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:59.012583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:11:59.013637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:11:59.014561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 183/362] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:11:59.020710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.020887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.021065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.030768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.031297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.031606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.032381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.033062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:59.033380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:59.035660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:59.044180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:11:59.045381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 180/363] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:59.046218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.046710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:11:59.113768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.142407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 184/364] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:11:59.142781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.157598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.187035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64047bea-0eed-48bd-bdfb-6075ed5ccedc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.238542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.239155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.239431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.239734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.240465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:59.240767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:59.242528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:59.248502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 181/365] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:59.256691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.271587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c25e5-7ae3-45cc-9b8e-22ca89c32b25 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.320458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.321299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.321738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.322165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.322999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:11:59.323636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:59.325219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:59.338633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 185/366] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:11:59.344554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.447926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.474923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d0cb8-6406-475b-b61f-7cb05caf9d61 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.526672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.527466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.527858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.529152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.536920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef974-60b5-48b3-ae46-484443bfe91e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.551789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.597673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.598495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.598754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.599130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.654292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:59.656150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:59.724238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:11:59.726258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:11:59.785665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.800663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 182/367] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:11:59.802023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.848601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 186/368] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:11:59.849553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:11:59.867902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:11:59.938167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2920-7f34-4e56-b0cd-2f673a25f0a5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.977061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:11:59.977844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:11:59.978214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:11:59.978759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:11:59.980018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc056f-26fc-4a02-b5a1-b91e13091302 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:11:59.990838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:11:59.998110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:11:59.999218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:00.000248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:00.006813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 183/369] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:00.007900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.029731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.031701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.033185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.033623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.036595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.038225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.040813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 187/370] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:11:59 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.041815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.065919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.159956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda6ed-c3cc-466d-a150-fc27b00e451e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.168589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.173874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b641f-d2c8-4d86-bd18-bb16c90ae66e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.202790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.203963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.204352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.204991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.206875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.208856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.211623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 184/371] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.212560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.219586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.220186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.220459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.220725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.221390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:00.221679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.223503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.236577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 188/372] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.237429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.352176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.362335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf37fa-d5dc-4d30-8f80-8241f5bb1252 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.382143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a89372-8556-4b72-8c0d-53b29df5a6ef None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.401520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.402262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.402547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.402927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.404430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.406064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.408450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 185/373] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.409152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.423816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.424574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.424856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.425124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.426807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.428502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.430231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 189/374] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.430991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.467921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.547255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d74a3f-ddfe-45ca-a04f-97fc27c5d2c7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.572161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8285f3-7da9-4949-9b35-7cd0928688c5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.594127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.594775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.595056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.595658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.618184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.619085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.619363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.619666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.620546 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=62067e6078ac4fe69c558de35c40ee8e) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:00.620862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.622603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.644302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 190/375] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.645678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:00.662749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.721600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.723022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.826451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:00.868557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d169d3-da93-43d4-8066-4ae093da8f76 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:00.915073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:00.915738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:00.916018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:00.916291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:00.917652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=62067e6078ac4fe69c558de35c40ee8e) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:00.917652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:00.920661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56918158-cb44-43be-81e3-ca5c423c8e7c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:00.933138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 186/376] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:00.937820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 191/377] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 260 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:00.939821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:01.063665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268bbcb-e22a-481d-96fd-91ceb3a1b01a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.064080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268bbcb-e22a-481d-96fd-91ceb3a1b01a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.064372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268bbcb-e22a-481d-96fd-91ceb3a1b01a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.065684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95b2f5-d6ce-4ab5-a572-57ac579d9f6d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:01.066126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 192/378] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:01.072823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.073101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.073384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.102452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:01.104424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:01.129510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:01.130165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.130455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.130723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] PATH_INFO: `/v3/projects/fc5a8e0f96e744888fe26890d52ecbca` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.131967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] RBAC: Authorizing `identity:get_project(project_id=fc5a8e0f96e744888fe26890d52ecbca)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:01.133448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:01.140536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 187/379] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:00 2022] GET /identity/v3/projects/fc5a8e0f96e744888fe26890d52ecbca => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:01.158311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:01.176035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:01.230724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 193/380] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:01.285094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6970a-c3f0-4987-8bb6-9d319e05cb7e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:01.324790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:01.325350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.325628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.325904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] PATH_INFO: `/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.331740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] RBAC: Authorizing `identity:get_group(group_id=f45b544ca7d54f7ba51dbfaa9ccbf5f2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:01.333201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:01.336151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 188/381] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:01 2022] GET /identity/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:01.341151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:01.343726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:01.448452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0ab85-1eac-48aa-bd87-b7e926070990 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:01.485841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:01.486455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.486735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.487010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.487778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] RBAC: Adding query filter params (group.id=f45b544ca7d54f7ba51dbfaa9ccbf5f2, role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=fc5a8e0f96e744888fe26890d52ecbca) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:01.488089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:01.489549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:01.499982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 194/382] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:01 2022] GET /identity/v3/role_assignments?group.id=f45b544ca7d54f7ba51dbfaa9ccbf5f2&scope.project.id=fc5a8e0f96e744888fe26890d52ecbca&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:01.705151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:01.794319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:01.822175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09321e7-9821-452f-a777-04b64fc36949 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:01.873214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:01.873853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:01.874134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:01.874407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:01.887785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-06b39caf-f9b2-475f-be55-77865bcab1d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:01.893250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:01.895259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:01.897244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 189/383] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:01.901928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:01.987875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:02.012091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1829df-7920-4755-ab19-ddb721e32db7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.048262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.048876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.049152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.049423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.050089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:02.050389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.051855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.057721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00392e-dce3-4fa9-8866-b61736a67b1f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.058001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00392e-dce3-4fa9-8866-b61736a67b1f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.058273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00392e-dce3-4fa9-8866-b61736a67b1f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.059910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 190/384] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:02.065243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 195/385] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:12:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:02.066756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.067280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.067280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.068583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.087956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:02.089148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:02.192877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 191/386] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:02.202974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4722c6-9f0e-4844-b654-06ddc9d8ef9e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.233037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:02.242267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.243360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.243981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.244457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.246400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.249233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.253162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 196/387] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:02 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:02.259195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.353043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:02.399523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9629ae-df9e-4243-8c0f-781cd414cf0e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.443412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.443986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.444260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.444528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.445660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.447086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.449027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 192/388] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:02 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:02.455131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.610380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ce8e7-f55a-45e4-acbc-5b7ff3a0c20f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.658565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.659225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.659500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.659803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.662285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, group_id=afed9ac6b7f94d03b2990fb9fa375f03, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:02.663730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:02.696901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 197/389] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 21:12:02 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:02.731575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:02.862013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798de57-bdf3-41cb-b1ca-18d9a24e44eb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:02.910305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:02.911307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:02.911825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:02.912376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:02.912898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:02.929454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:02.936018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:02.937082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:02.937949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:02.945659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 193/390] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:02.946995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:03.018180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:03.066522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8278-2254-4c34-9aef-8fbe846cd6fd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:03.107025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:03.108433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.108940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.109443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.110371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:03.110877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:03.112460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:03.121319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 198/391] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:03.126935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:03.248722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266bbc8-d875-4f15-b8bf-d180c38d2be0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:03.296459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:03.298012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.298498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.298997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.300469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:03.302116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:03.304277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 194/392] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:03 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:03.309379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:03.420465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2e554-20f1-4564-9ca8-4e728a6e0dfd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:03.440073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:03.458436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:03.459428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.459951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.460446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.461795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:03.463430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085c5f2-af14-4bd9-ba06-638c578589e7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:03.465677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 199/393] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:03 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:03.471736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:03.535865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:03.583949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e3ce-0c62-41dd-8284-8caf453f85cc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:03.624049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:03.624715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.624994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.625343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.628448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=807d56e26a214f4f89dc4c31bf994c3b, group_id=afed9ac6b7f94d03b2990fb9fa375f03, role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:03.629969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d0abd-5625-4256-a39a-499a15b1a9f9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:03.649253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 195/394] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 21:12:03 2022] PUT /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/da0177264e704fa19ac1889c21e21eb5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:03.913819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0163-bbe7-4dbf-920f-306724d7d52a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.914341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0163-bbe7-4dbf-920f-306724d7d52a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.914762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d0163-bbe7-4dbf-920f-306724d7d52a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.916637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 200/395] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:03.917086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacf903-8fa8-4dc6-aa51-2a1b580946e2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.917521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacf903-8fa8-4dc6-aa51-2a1b580946e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.917799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacf903-8fa8-4dc6-aa51-2a1b580946e2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.919602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 196/396] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:03.921463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.921959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.922490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.924980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:03.925261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:03.925534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:03.945504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:03.947074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:03.947282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:03.948727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:04.025886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 197/397] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:04.034234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 201/398] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:04.343419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.403358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:04.462298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79192a7b-0f13-4f87-936b-4766f300019b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:04.465428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.502149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:04.503319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.503802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.504243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.515616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:04.521482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:04.522568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:04.523583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:04.530601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 198/399] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:04.532444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.649293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a881-0aa7-4fd6-b9a8-5a7ae639e19b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:04.670983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:04.692984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da3caa-b14b-4c07-bf12-df07e043c5c3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:04.696758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:04.697626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.698102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.698562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.699406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:04.699932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:04.701497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84d6c9-5a88-45ab-810a-372f1f547298 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:04.714910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 199/400] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:04.715132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:04.716657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c038-e21f-4960-91ac-17fdbbede2d8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.717112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c038-e21f-4960-91ac-17fdbbede2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.717552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13c038-e21f-4960-91ac-17fdbbede2d8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.719573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 200/401] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:04.720862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13c038-e21f-4960-91ac-17fdbbede2d8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.741062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:04.741749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.742029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.742287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.755306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c3905bbe-96bb-4066-9159-56c27689b539 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:04.761006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:04.762053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:04.763025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:04.766676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 202/402] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:04.768076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.768533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.768954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.792824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:04.794431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:04.839701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13c038-e21f-4960-91ac-17fdbbede2d8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:04.877726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:04.878765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:04.879583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:04.879975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:04.881523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:04.883081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:04.884427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 203/403] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:04.891383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 201/404] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:04.891383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.892895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:04.973502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.091152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1065bd-3a54-430e-8903-164fe635ead7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.097445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ad54e-09f7-489f-aa2c-07c6724898e5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.098522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.130246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.131353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.131716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.132004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.133194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.134805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.136646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 204/405] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.143361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.183792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.184412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.184692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.184962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.185636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:05.185938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.187313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.208058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 202/406] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:12:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.208911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.254526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1b3e-ef7f-4484-9380-14e2adba9df6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.295454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.296151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.296425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.296699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.299057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, group_id=afed9ac6b7f94d03b2990fb9fa375f03, role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.300512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.316362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 205/407] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 21:12:05 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/da0177264e704fa19ac1889c21e21eb5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:05.324883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.325789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee61733-8821-4c02-b02f-34086ca3057f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.327790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.372379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.373048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.373302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.373533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.382133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.388633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7677b828-52dd-4b64-89a7-5277c646d2ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:05.397806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:05.398767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 203/408] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:05.404840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45934909-6482-497b-9d18-6ace357d7ad4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.405724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45934909-6482-497b-9d18-6ace357d7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.405724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45934909-6482-497b-9d18-6ace357d7ad4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.405724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 204/409] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:05.408650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45934909-6482-497b-9d18-6ace357d7ad4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.487458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec838f9-1bb9-451f-a6e6-a0188c85e682 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.542991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.544002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.544691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.544970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.546664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.549203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17a9d6-412b-4b30-b909-d6fc23a930c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.551803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 206/410] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.552934 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.553318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.553694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.554791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45934909-6482-497b-9d18-6ace357d7ad4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.580625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:05.582662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:05.596991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.597689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.598087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.598436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.599220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:05.599631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.601164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.610446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 205/411] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.611632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.643467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.675741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 207/412] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:05.676401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.718380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.745064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b42d-693d-472e-b9ff-471f1d023689 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.792923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.793562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.793842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.794119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.795927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.798244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.800923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 206/413] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.802156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4393b05-9a04-4b61-8504-88bc3c5d9f50 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.815160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.869321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.870286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.870684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.871054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] PATH_INFO: `/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:05.875968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] RBAC: Authorizing `identity:get_group(group_id=f45b544ca7d54f7ba51dbfaa9ccbf5f2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:05.877611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:05.879747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 208/414] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2 => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:05.885182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:05.920053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:05.936528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9512aa-5f8e-4836-a392-af1b169db6b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:05.993033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:05.993713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:05.993995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:05.994270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.002856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] RBAC: Authorizing `identity:create_grant(project_id=62067e6078ac4fe69c558de35c40ee8e, group_id=afed9ac6b7f94d03b2990fb9fa375f03, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:06.004362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:06.025647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 207/415] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 21:12:05 2022] PUT /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e/groups/afed9ac6b7f94d03b2990fb9fa375f03/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:06.026469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:06.097543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:06.104818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedee8b-dc0b-481e-9570-d99f0c87260b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:06.176314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:06.176963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.177238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.177505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] PATH_INFO: `/v3/projects/fc5a8e0f96e744888fe26890d52ecbca` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.180024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] RBAC: Authorizing `identity:get_project(project_id=fc5a8e0f96e744888fe26890d52ecbca)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:06.181479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:06.184339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 209/416] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:05 2022] GET /identity/v3/projects/fc5a8e0f96e744888fe26890d52ecbca => generated 344 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:06.189719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:06.232731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a9ca4-3149-4224-b646-ae6fc5ecb6dd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:06.308524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:06.309431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.309767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.310051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.330679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:06.336182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:06.337064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:06.337986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 208/417] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:06.339459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ee3bb-6a38-4df4-97d4-2fd67a66586c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:06.345089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:06.364553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:06.402407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:06.403495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.403807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.404074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] PATH_INFO: `/v3/projects/fc5a8e0f96e744888fe26890d52ecbca/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.412089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc5a8e0f96e744888fe26890d52ecbca, group_id=f45b544ca7d54f7ba51dbfaa9ccbf5f2, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:06.413593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbaf1c9-dd76-4bcb-9996-98d748d4a8b4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:06.443047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 210/418] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 21:12:06 2022] PUT /identity/v3/projects/fc5a8e0f96e744888fe26890d52ecbca/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:12:06.443988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4242e1-6cd0-4e1f-af0c-395cdafb1354 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.444274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4242e1-6cd0-4e1f-af0c-395cdafb1354 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.444553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4242e1-6cd0-4e1f-af0c-395cdafb1354 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.445818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 211/419] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:06.451829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.452172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.452402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.472469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:06.473793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:06.475032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc09d8-cdcf-472d-8694-f73574b2515f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:06.509132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:06.525366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:06.526235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.526655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.527027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.527993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:06.528702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:06.530524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:06.541933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 209/420] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:12:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:06.548324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:06.573840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 212/421] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:06.643551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:06.670897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27801397-1ef8-4d8c-ac87-66905851cdd2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:06.710386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:06.710977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:06.711324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:06.711629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:06.785813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:06.787314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:06.864637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:06.883378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:06.892026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 210/422] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:12:06.900498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:07.006408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a5be-76c6-409a-87e2-0d638cf7756c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.055067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.057036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.057314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.057587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.058899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:07.064301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:07.064301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 211/423] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:06 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:07.071565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:07.118901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc247-8aa1-4b5e-a087-694d77193b93 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.159898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:07.192405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.193043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.193322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.193593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.214865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:07.220491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:07.221418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:07.222232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 213/424] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:07.230766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:07.253767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac351f6-90e2-48f4-a479-44c3bc7c8e2a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.275985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:07.294878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.295996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.296456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.297623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.299191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:07.301095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:07.303152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 212/425] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:07 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:07.309702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:07.481748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd18b4-b6a3-4c18-b86f-9c9cadfc5fb9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.503770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bfe10-b2da-4a45-acba-122f5654db68 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.528735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:07.558415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.559679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.560165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.560609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.561458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:07.562663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:07.564930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:07.581192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.582090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.582545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.582995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.584035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:07.585104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 214/426] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:07.585590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:07.587221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71243-a7d4-4b7e-acdf-504fe38bf07e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:07.596055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:07.610360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 213/427] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:07 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 611 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:07.714335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:07.739429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0a28-c2c9-4739-8e69-dc4ce2876ba3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:07.782369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:07.783147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.783824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.784198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.858015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:07.859505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:07.889640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0cf12-233c-4ee6-8e10-75aef8adae68 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.889956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0cf12-233c-4ee6-8e10-75aef8adae68 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.890196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0cf12-233c-4ee6-8e10-75aef8adae68 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.892627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 214/428] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:07.898045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:07.898324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:07.898596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:07.951202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:07.951843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:07.953651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:07.973267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 215/429] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:07.978564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.065558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 215/430] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:08.111857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f314-b801-4f08-8a69-e97cb602c0d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.151053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.152060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.152535 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.153283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] PATH_INFO: `/v3/projects/807d56e26a214f4f89dc4c31bf994c3b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.154612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] RBAC: Authorizing `identity:get_project(project_id=807d56e26a214f4f89dc4c31bf994c3b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:08.156412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:08.158811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 216/431] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:07 2022] GET /identity/v3/projects/807d56e26a214f4f89dc4c31bf994c3b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:08.164240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.293197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16845b69-cb63-4136-99df-8138f1d3da5d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.295651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:08.339094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.339697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.339973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.340244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.341340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:08.342777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:08.344803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 216/432] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:08 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:08.350346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.378040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.423901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:08.474520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9c297-e574-48ba-9e7c-338b1e850e00 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.514778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58fb47-6404-4f07-a90f-359420d11e42 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.517798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.518405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.518678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.518953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.520381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=807d56e26a214f4f89dc4c31bf994c3b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:08.520690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:08.522136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:08.528765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 217/433] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:08 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=807d56e26a214f4f89dc4c31bf994c3b&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:08.568950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.569663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.569986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.570267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.583962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-086f1fe5-88cc-44b9-a971-10bca40c26be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:08.589379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:08.590301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:08.591123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 217/434] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:08.597717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.678354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85120-bd27-4259-9681-86332fe45736 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.678672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85120-bd27-4259-9681-86332fe45736 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.678904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff85120-bd27-4259-9681-86332fe45736 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.680535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 218/435] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:08.688893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.689230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.689438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.717914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:08.720445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:08.732690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca18376-9375-4781-bced-70f6ea94b391 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.748197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:08.761826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:08.778395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.779567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.779567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.779567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:08.780098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:08.780404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:08.782407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:08.794399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 218/436] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:08.804140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:08.823236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 219/437] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:08.919692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b09f5-12be-47e5-969e-4442485f2d79 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:08.957240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:08.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:08.958124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:08.958384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.002854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.031655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.033155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.107879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.147975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 219/438] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:09.149313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.216562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f331358-5991-432f-9678-04a8fd068e4a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.228834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.250503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.251339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.251641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.251937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.255677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf77191-5e87-4149-b44d-9072e0f319d7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.261678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5be21040-7348-4018-ba44-f46d7b1efdfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:09.267206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:09.268346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:09.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 220/439] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:09.269827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f0eb7-d94c-4ee3-9507-a4b994e0b1ad None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.270081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f0eb7-d94c-4ee3-9507-a4b994e0b1ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.270280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f0eb7-d94c-4ee3-9507-a4b994e0b1ad None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.271612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 221/440] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:09.273970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f0eb7-d94c-4ee3-9507-a4b994e0b1ad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.291887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.292515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.292802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.293076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.294194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.295813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8eb7-a845-4b39-b56d-44b41b02074f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.298063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 220/441] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:09.299067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.299241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.299524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.299820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.317899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:09.319078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:09.378957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f0eb7-d94c-4ee3-9507-a4b994e0b1ad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.385206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 221/442] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:09.386082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.413998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.414853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.415136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.415412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.416113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:09.416416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.417786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.426989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 222/443] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:09.432645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.472151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.503150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a8466-f375-4d08-b0dc-3839a1045e3a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.524093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.539342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.540000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.540303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.540533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.541633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.543051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.543864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6839-a29e-4d01-b2d4-326b9450774f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.544811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 222/444] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:09.550274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.585580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.585796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.586033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.586309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.658717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81099a-a023-4b0f-b76b-4391c2347706 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.667302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.668802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.691778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.692377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.692648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.692914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.693629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=62067e6078ac4fe69c558de35c40ee8e) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:09.693930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.695330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.701022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 223/445] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:09.701623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.713823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.761844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:09.783710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 223/446] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:12:09.789705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.830789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e3224-2134-45ae-9087-d04788b7a4b8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.869870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.870508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.870790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.871060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.884483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:09.889850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:09.890980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:09.891821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 224/447] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:09.897667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.907987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb6f37-d25c-401b-b852-f8f5f752dd98 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:09.951938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:09.952556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:09.952848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:09.953311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] PATH_INFO: `/v3/projects/62067e6078ac4fe69c558de35c40ee8e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:09.954508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] RBAC: Authorizing `identity:get_project(project_id=62067e6078ac4fe69c558de35c40ee8e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:09.955998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:09.957600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 224/448] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/projects/62067e6078ac4fe69c558de35c40ee8e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:09.963088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:09.967003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.006310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c52b07-fc34-4783-88c8-4b7077a00ec3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.016215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.039996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.040591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.040866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.041130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.041781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:10.042078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.043418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.051873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 225/449] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:10.057482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:10.064580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22e49-3147-4989-9430-68cc907a05eb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.097772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.098404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.098681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.099042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] PATH_INFO: `/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.100191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] RBAC: Authorizing `identity:get_group(group_id=afed9ac6b7f94d03b2990fb9fa375f03)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.101626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.103178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 225/450] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:09 2022] GET /identity/v3/groups/afed9ac6b7f94d03b2990fb9fa375f03 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:10.108961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:10.159656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e897c-4427-497f-8d0b-a929cc771715 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.184336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.193375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.193981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.194259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.194531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.207608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447ff99-82a9-40ef-b540-7be682ba94da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.227492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.241957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.242555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.242828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.243093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.243846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] RBAC: Adding query filter params (group.id=afed9ac6b7f94d03b2990fb9fa375f03, role.id=869347abd4b848d1abcc7f9e6ffc94a9, scope.project.id=62067e6078ac4fe69c558de35c40ee8e) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:10.244148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.245558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.251737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 226/451] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:10 2022] GET /identity/v3/role_assignments?group.id=afed9ac6b7f94d03b2990fb9fa375f03&scope.project.id=62067e6078ac4fe69c558de35c40ee8e&role.id=869347abd4b848d1abcc7f9e6ffc94a9 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:10.262101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.263616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.371798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 226/452] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:12:10.378522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:10.481645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d87532-45fd-45c7-aaf5-0027e614c1b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.517012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.517650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.518021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.518303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] PATH_INFO: `/v3/projects/fc5a8e0f96e744888fe26890d52ecbca` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.520014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] RBAC: Authorizing `identity:get_project(project_id=fc5a8e0f96e744888fe26890d52ecbca)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.522250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.524398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 227/453] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:12:10 2022] GET /identity/v3/projects/fc5a8e0f96e744888fe26890d52ecbca => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:10.530794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:10.558543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.634811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cef88-4e11-4313-aec3-b460459aebb9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.669838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.670486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.670724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.670989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] PATH_INFO: `/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.672129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] RBAC: Authorizing `identity:get_group(group_id=f45b544ca7d54f7ba51dbfaa9ccbf5f2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.673665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79085c7-3feb-4a56-9957-ade7e1dab503 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.675216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 227/454] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 21:12:10 2022] GET /identity/v3/groups/f45b544ca7d54f7ba51dbfaa9ccbf5f2 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:10.681092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:10.738354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:10.787707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a74dcf-9d1b-4ecf-b9d8-c438cead56f4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:10.824369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:10.824964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:10.825243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:10.825516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:10.826241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] RBAC: Adding query filter params (group.id=f45b544ca7d54f7ba51dbfaa9ccbf5f2, role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=fc5a8e0f96e744888fe26890d52ecbca) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:10.826545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:10.828605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff8f22-39f6-4fb5-a54c-bf93db95f9dc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:10.834481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 228/455] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 21:12:10 2022] GET /identity/v3/role_assignments?group.id=f45b544ca7d54f7ba51dbfaa9ccbf5f2&scope.project.id=fc5a8e0f96e744888fe26890d52ecbca&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:13.470909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacaa1b-fe4b-4c66-b81c-9148550ee5ff None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.471988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacaa1b-fe4b-4c66-b81c-9148550ee5ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.472479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacaa1b-fe4b-4c66-b81c-9148550ee5ff None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.474411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 228/456] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:13.479953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383356-75eb-45ad-a612-1d1aa1adbc3f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.480915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383356-75eb-45ad-a612-1d1aa1adbc3f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.481435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383356-75eb-45ad-a612-1d1aa1adbc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.508758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32383356-75eb-45ad-a612-1d1aa1adbc3f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:13.510370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32383356-75eb-45ad-a612-1d1aa1adbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:13.600824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 229/457] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:13.612954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68d57-a812-4005-9bc8-95af309e7a11 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.613235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68d57-a812-4005-9bc8-95af309e7a11 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.613512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68d57-a812-4005-9bc8-95af309e7a11 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.614948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 229/458] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:13.620323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039823-29de-4d3b-84fd-4a05c179cb9c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.620878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039823-29de-4d3b-84fd-4a05c179cb9c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.621369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a039823-29de-4d3b-84fd-4a05c179cb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.642598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3a039823-29de-4d3b-84fd-4a05c179cb9c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:13.643847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a039823-29de-4d3b-84fd-4a05c179cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:13.717897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:13.748190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 230/459] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:13.801207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb33f5a-0f8b-4c9c-ac66-076ce4e260d5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.801544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb33f5a-0f8b-4c9c-ac66-076ce4e260d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.801751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb33f5a-0f8b-4c9c-ac66-076ce4e260d5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.803071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 230/460] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:13.808250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:13.808538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:13.808872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:13.833298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:13.834786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:13.918719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb33f5a-0f8b-4c9c-ac66-076ce4e260d5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:13.930125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 231/461] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:14.055305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.057148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.066229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb33f5a-0f8b-4c9c-ac66-076ce4e260d5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.130663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.131300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.131598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.131935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.154603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.156699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:14.179896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:14.180907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:14.182373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:14.182373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.182373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:14.182373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 231/462] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:14.187399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.231075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ffd30-256b-4206-954f-3cf13f9441b1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.242671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.304226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.305218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.305776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.306249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.317818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.319555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.335051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abec4db-4ef7-4133-86b2-87e02226dcce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:14.336569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:14.337615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:14.338461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:14.338461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.338461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:14.345292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 232/463] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:14.346195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.378688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.379713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.380090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.380456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.381733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:14.382259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.383737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.399713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 232/464] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:14.400841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.416284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.465970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9900cc-6b5f-401c-8018-d5372053a836 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.502331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.503773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.504458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.504930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.516422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.518168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:14.534054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:14.534997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:14.535895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:14.535895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:14.535895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:14.537942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 233/465] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:14.538974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.548011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52730df-39d9-466c-b030-8bb2b682a749 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.558042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.588633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.589243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.589523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.589794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.590464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:14.590763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.592167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.604796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 233/466] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:14.605852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.661074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c7149-a583-4dee-b503-db1f995f52d8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.689672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.696145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.696756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.697036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.697309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.697954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.699399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311b31b-7075-4ce9-803c-da4a57aee6e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.721389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 234/467] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:14 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:14.722848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabba07-8e43-4da5-bf15-60b1552debf0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.723319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabba07-8e43-4da5-bf15-60b1552debf0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.723781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabba07-8e43-4da5-bf15-60b1552debf0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.729491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 235/468] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:14.730652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabba07-8e43-4da5-bf15-60b1552debf0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.743525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6db6ea-3f2a-4c6e-a78f-fc9662d736d5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.787836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.788662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.789042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.789419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.790199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:14.790606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.792216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e2019-19f5-4f91-a238-284e61fd12ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.810303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 234/469] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:14.814286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.814775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.815218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.831002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:14.844253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:14.845710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:14.878536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabba07-8e43-4da5-bf15-60b1552debf0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:14.920680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:14.922329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:14.922623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:14.922913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:14.923930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:14.925249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:14.938482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 235/470] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:14.939332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:14.957315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 236/471] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:14 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:15.024993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:15.080120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b97c2-ec48-41fc-8b26-1f71aa3324ff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:15.120010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:15.120643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.120924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.121197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.121924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:15.123465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c40f4-a704-48b8-a362-6bfe25411500 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:15.156718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 236/472] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:14 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:15.248208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:15.297639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6227ba-8bea-40cb-8eb6-0b92403112fd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.297967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6227ba-8bea-40cb-8eb6-0b92403112fd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.298194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6227ba-8bea-40cb-8eb6-0b92403112fd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.299911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 237/473] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:15.306137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.306464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.306693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.335318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:15.337115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:15.368621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca990fc-cf5f-486a-8083-7b95fd3208b8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:15.406688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:15.424290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:15.425197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.425476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.425749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.435099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 238/474] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:15.440997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:15.442452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:15.461357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:15.462314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:15.463176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:15.463176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:15.463176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:15.463176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 237/475] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:15.470285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:15.517770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:15.581692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd46-97e9-4f17-ab21-cdf10f70ac4e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:15.623292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:15.623856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.624131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.627430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.627430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:15.627430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:15.627430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:15.645255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 239/476] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:15.653160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:15.784630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:15.845762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bd3ad-c096-4eae-b7b2-b0bf86a4a785 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:15.943965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:15.945401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:15.946142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:15.946911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:15.948406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:15.954183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:16.016011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 238/477] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:15 2022] POST /identity/v3/users => generated 249 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:16.045766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7e93e-3432-415b-9adc-fcb3008eaae9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:16.048031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:16.098619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:16.099713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:16.100228 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:16.100700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:16.101666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:16.103266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:16.117010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 21:12:16.117010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 21:12:16.117010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [parameters: {'id': '5fccb7158883434bacec851d688f6b65', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 21:12:16.117010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 09 21:12:16.139849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:12:16.141229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 21:12:16.142148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 21:12:16.143017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:12:16.143997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:12:16.144833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': '5fccb7158883434bacec851d688f6b65', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:16.145633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 21:12:16.146442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:16.147276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 240/478] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 21:12:15 2022] POST /identity/v3/roles => generated 144 bytes in 357 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:16.156330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:16.210242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:16.283890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec849-f664-4ff4-9582-74fb0d389ea8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:16.318338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:16.318958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:16.319279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:16.319579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:16.331112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7fc13194-cd9c-4f3e-8cb9-a523fa8365b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:16.336575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:16.337546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:16.338516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 239/479] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:16.346209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:16.454307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae0fd5-ddbd-4fea-bad3-86291d2b4b47 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:16.489430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:16.490369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:16.490849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:16.491318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:16.492221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:16.492742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:16.494285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea4f8-34c9-4446-b5b3-94cca225f028 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:16.507670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 241/480] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:17.622273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2130e8-e21f-4b98-912e-2721f486a523 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:17.622833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2130e8-e21f-4b98-912e-2721f486a523 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:17.622833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2130e8-e21f-4b98-912e-2721f486a523 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:17.630789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 240/481] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:17.643064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9c1-1a59-4f8d-b260-5968ef0b47d1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:17.643562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9c1-1a59-4f8d-b260-5968ef0b47d1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:17.643671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb9c1-1a59-4f8d-b260-5968ef0b47d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:17.676580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e9aeb9c1-1a59-4f8d-b260-5968ef0b47d1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:17.677783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e9aeb9c1-1a59-4f8d-b260-5968ef0b47d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:17.706945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:17.771908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 242/482] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:17.775627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8892-ca48-4d75-90c8-71d82d4f7ee2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:17.776063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8892-ca48-4d75-90c8-71d82d4f7ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:17.776481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8892-ca48-4d75-90c8-71d82d4f7ee2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:17.778410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 241/483] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:17.784993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebe31c-420e-426d-a6c2-ddba78e4ae72 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:17.785455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebe31c-420e-426d-a6c2-ddba78e4ae72 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:17.785894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebe31c-420e-426d-a6c2-ddba78e4ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:17.804780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bbebe31c-420e-426d-a6c2-ddba78e4ae72 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:17.806169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bbebe31c-420e-426d-a6c2-ddba78e4ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:17.897090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 243/484] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:18.151052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bb93-90c4-410a-a998-fa7432396fb8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.151249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bb93-90c4-410a-a998-fa7432396fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.151504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bb93-90c4-410a-a998-fa7432396fb8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.153361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 242/485] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:18.160291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6bb93-90c4-410a-a998-fa7432396fb8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.163133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.163404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.165305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.201612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:18.203335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:18.315928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:18.316123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 244/486] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:18.317370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.321396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6bb93-90c4-410a-a998-fa7432396fb8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.350879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:18.365227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.366744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.367255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.367776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.394821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:18.400215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:18.401193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.402018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:18.403817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 243/487] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:18.413346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.510857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f59ec-72df-482a-bb2b-f4c4d9ab5628 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.566828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.567664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.567959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.568289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.580031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:18.585931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:18.586811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.587631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:18.591693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 245/488] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:18.593912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.617737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19100f63-9df9-40b4-af53-d8ac5b78da1f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.683181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.683871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.684093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.684384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.685100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:18.685407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:18.687412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:18.701339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 244/489] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:18.702381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.718112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:18.719966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:18.744217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b3f2-298f-4116-ab90-179c1c748c65 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.781930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.782857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.783313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.783809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.784653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:18.785148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:18.786698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:18.795714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 246/490] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:18.797670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.840295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fc188-2714-4759-b1a3-7a2c070e6f10 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.896442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.897463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.897747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.898085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.913796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:18.914364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77731a4-8424-4cb7-aa63-ea345fa3a60f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:18.920567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:18.921876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:18.922758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 245/491] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:18.923426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:18.961048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:18.962039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:18.962541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:18.963010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:18.974204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:18.977402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:18.979341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:18.994904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:18.995882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:18.996723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:18.996723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:18.996723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:18.998761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 247/492] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:19.000285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.012206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.044478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de5343-b393-43cc-a784-b3e4ae09083c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.084076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.084705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.085003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.085280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.097867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.099339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-37b76beb-7d33-4709-9c55-e20f08ba8aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:19.114886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:19.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:19.117060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:19.117060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:19.117060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:19.117060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 246/493] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:19.117543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69a03-6406-47fc-9757-b634e4ae3405 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.117834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69a03-6406-47fc-9757-b634e4ae3405 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.118110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69a03-6406-47fc-9757-b634e4ae3405 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.118449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a425-dec4-4c1b-b4bf-c54a7ebfe1d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.123383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 247/494] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:19.124877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c69a03-6406-47fc-9757-b634e4ae3405 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.157389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.158192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.158798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.159187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.159779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:19.160088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.161458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.169563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 248/495] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:19.170370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.231113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.240864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c69a03-6406-47fc-9757-b634e4ae3405 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.267955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.279678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.280307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.281135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.281410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.282143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:19.282371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.283808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df04fe-5a29-46f9-a337-7f2a558a9324 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.284004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b551ec-e71f-4230-8d40-448ada6e0ba2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.303625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 248/496] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:19.304839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.305130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.305405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.325602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.326212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.326500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.326793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.327447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:19.327773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.329145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.331301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:19.332677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:19.341378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 249/497] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:19.342205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.413214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 249/498] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:19.413885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.461089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e416-296f-4e1b-8be3-b1123d6f139a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.478032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.498742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.499328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.499629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.499897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.509817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.511248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.522659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ab12-4b89-41b6-a4dc-3b1d674d19a8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:19.525995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:19.526901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:19.528034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:19.528034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:19.528034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:19.528034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 250/499] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:19.528730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.541752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.564524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.565157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.565441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.565720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.572934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.574385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c1682c0e-19b6-4a12-84fe-12477b2570da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:19.587157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:19.588183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:19.589337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:19.593007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 250/500] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:19.594005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe9f5-0c70-47b3-9ec3-2958e03bf177 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.594295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe9f5-0c70-47b3-9ec3-2958e03bf177 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.594577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe9f5-0c70-47b3-9ec3-2958e03bf177 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.595845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 251/501] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:19.596542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fe9f5-0c70-47b3-9ec3-2958e03bf177 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.654051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360200-9e8b-46ce-9aad-c7b9ca2c6bfd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.689882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.690507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.690785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.691059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.698428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.699895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.707588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fe9f5-0c70-47b3-9ec3-2958e03bf177 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:19.712141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:19.713186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 251/502] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:19.714131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.741742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.746232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.746836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.747115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.747384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.748048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:19.748349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.750589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e8a58-8d01-42b2-beed-60654cdebc08 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.761977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 252/503] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:19.763268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.763582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.763859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.781251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:19.782436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:19.808337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:19.816536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46dbfa-bdc1-4e58-a350-928588235d47 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.848098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 253/504] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:19.848929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.849963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:19.850561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:19.850841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:19.851113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:19.851872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:19.852175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:19.853599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:19.895897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 252/505] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:19.895897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:19.973143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655b03b-f725-420f-badd-5c0c1037b9e5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:19.983013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.002255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c49b5-b3ca-4c54-a5bd-aa3c8ee77c3c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.009168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.009984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.010261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.010538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.021235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:20.026065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:20.026972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 254/506] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:20.034247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.044147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.044756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.045039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.045311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.046657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:20.046964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.049154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.059658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.066432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 253/507] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:20.067236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.149169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59718c0-4d79-480b-b344-55483a87ecb6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.172588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4ddad-8c40-4587-8672-c39d316e8c9d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.183667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.184292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.184571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.184841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.192294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.193747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:20.205299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:20.206258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:20.207197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 255/508] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:20.207808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.207959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.207959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.208084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.223814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.279091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.281110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.294148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.310203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44bb8e-80ff-4866-95ac-c4445caedd57 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.343511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.344138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.344416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.344692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.345346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:20.345643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.346999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.354722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 256/509] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:20.355509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.379280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 254/510] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:20.380359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.423525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.467287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2f6a-ed83-4dd1-ba8c-608193d3f91f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.479793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4facefe-3359-4ba2-b80a-7ef4a28e6c2c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.500351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.500938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.501213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.501484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.513229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.514159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.514440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.515382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.518753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.525878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.527317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:20.542757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:20.543674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:20.544554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:20.544554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.544554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:20.544554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 255/511] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:20.544554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.571840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.573286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.638791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.647341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628e87d-ae15-4a66-a5b8-5a8529541173 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.676212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 257/512] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:20.677241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.681257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.681868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.682150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.682422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.683151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:20.683455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.684908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.707648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 256/513] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:20.707648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.729023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.782798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9e24-b2c0-4392-b2b3-bae59dc7b31f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.801218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd0b71-8554-4182-95b5-b3eca983ee61 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.816283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.816901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.817177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.817449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.827963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.829527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.835903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.836519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.836794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.837061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:20.843965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:20.844911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:20.845780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:20.845780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.845780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:20.845780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 258/514] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:20.845780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.852371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.853812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:20.869388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:20.870279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:20.871142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:20.871142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:20.871142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:20.871142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 257/515] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:20.871356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:20.876628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.944932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:20.948775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f422-ca7d-465a-864c-b1dc1aa9e58c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.973486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202ee8-d374-4b3d-8ed6-3bde03dede73 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:20.982488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:20.983074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:20.983375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:20.983669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:20.984304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:20.984598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:20.985965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:20.997320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 259/516] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:20.997928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.011398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.012049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.012322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.012601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.024305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.025736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:21.040758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:21.041743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:21.042631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:21.042631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:21.042631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:21.042631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 258/517] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:21.042631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.100240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.100409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca23bf-1dc7-43bd-903a-f5f8eadd5918 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.134912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.135512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.135819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.136093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.143812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5415fd-dcc3-475e-9d7e-8847f49addf1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.162336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.177327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.177927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.178205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.178476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.179117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:21.179438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.180809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.193078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 259/518] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:21.193897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.204191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.205649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.290451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2fcf-ed26-453a-b83a-4cbbd0fb2ced None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.304611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 260/519] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:21.304611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.309663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.326808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.327454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.327774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.328038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.328675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:21.328975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.330326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.341325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 260/520] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:21.342106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.364252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.406786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cf12a-fd6c-4b1a-b100-012c4b7df48e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.441896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.442503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.442782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.443051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.443711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.445157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.445256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88e4ec-f935-402e-89c2-03d6032cfbdd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.466632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 261/521] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:21 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:21.467440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.481290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.481920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.482265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.482537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.483224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:21.484332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.485896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.500906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 261/522] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:21.501985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.540849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.590561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e3f33-f31f-4672-af66-1b695122b844 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.605516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.618256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5850c-b372-4428-bf77-206caa040c7f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.633790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.634484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.634704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.634973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.646485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.647981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:21.660789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:21.661709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 262/523] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:21.663972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.665172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.665805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.666082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.666353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.678549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.680538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:21.697780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:21.698742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:21.699658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:21.699658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:21.699658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:21.699658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 262/524] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:21.699895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.782918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729cded-7fbb-4ae6-bc08-c490585b4d0d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.802728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.817865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.818493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.818774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.819046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.819145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7c7aa-02d4-4a53-b135-dd06ddc9bcfb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:21.825650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.827150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:21.838893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:21.844734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 263/525] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:21.845763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.862051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:21.862686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:21.863082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:21.863358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:21.870996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:21.872665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:21.881383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:21.888871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:21.890545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:21.891413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:21.891881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 263/526] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:21.892572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:21.963404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686f6a-ccf3-41d2-afe8-3471fb89cbe7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.000627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.001242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.001526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.001804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.002547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.002856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.004909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.036044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14192-7834-4e19-91ce-92c49675f8ef None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.056349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 264/527] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.057217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.083070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.084419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.084705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.084979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.086040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.086726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.087880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.111419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 264/528] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.111419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.125571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.177772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.188792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737b28-b66d-47ec-8ca4-1a12587471b8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.230795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.231446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.231760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.232040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.233166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.233709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.235319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.243918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713790b-56b3-4013-bb31-ecd17f42ba3c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.262474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 265/529] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.263996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.292141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.292918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.293397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.293871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.295573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.295917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.299215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.325944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 265/530] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.326867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.398691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36e9e-d163-41f5-9ee3-781a460b4246 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.416583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.444647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.445579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.445855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.446121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.471946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebd887-1d55-4abc-b7e5-8e9d78aff5cd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.474113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.516289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.516928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.517205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.517475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.524941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.526575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.536261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.538963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:22.542464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:22.543366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:22.544301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 266/531] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:22.545393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.680198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c5eac-255a-488a-9c6b-6f4f14922468 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.681815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 266/532] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:22.682511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.717280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.727374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.728320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.728599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.728872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.729608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.729915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.731378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.743589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 267/533] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.743589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.767624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:22.846037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932670a-6027-4f96-86b0-71b4b2a8e2be None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.885979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc403-8633-4953-9381-07c5e4be905a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:22.894880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.895793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.896254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.896743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.897651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8985883a056c431b885bb07303d3f8af) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.898207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.899883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.907793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 267/534] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/role_assignments?user.id=8985883a056c431b885bb07303d3f8af&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.909115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:22.938419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:22.938976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:22.939320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:22.939618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:22.940424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:22.940727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:22.942339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:22.969481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 268/535] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:22.978979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:23.046517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:23.055000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc723b-90d9-4158-9232-c18ac75e4744 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:23.087930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:23.115860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74878559-c20d-4767-b5d0-187b275a0187 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:23.132291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:23.132944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:23.133228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:23.133502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:23.152955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:23.156517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:23.166629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:23.167572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:23.168066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:23.168567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:23.169518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=b670a7afd2264609894dec2feabc4581) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:23.170070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:23.172716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:23.179458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 269/536] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/role_assignments?user.id=b670a7afd2264609894dec2feabc4581&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:23.193763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:23.194722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:23.195679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:23.195679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:23.195679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:23.195679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 268/537] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:23.203905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:23.332868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003b69-a001-40e3-b723-798e34f942e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:23.351006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:23.371557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:23.372526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:23.373000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:23.373461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:23.374904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:23.375398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:23.376989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:23.390338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 270/538] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:23.395613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:23.530692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ade20-1708-4f05-849a-4dee879214f8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:23.580263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:23.580912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:23.581195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:23.581470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:23.592419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:23.593976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:23.607940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:23.609368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:23.610311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 269/539] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:23.615344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:23.760067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44130f0b-510c-4751-908c-7eca31d7d15a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:23.797423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:23.797953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:23.798232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:23.798503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:23.799198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:23.799502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:23.800952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3faba-4651-4139-9b50-ecae9dcd2b81 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:23.813913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 271/540] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:23.819722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:23.876449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:23.965368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668de30-8039-4848-a320-2c66baa4c85e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:24.018888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:24.019513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:24.019819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:24.020093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:24.020850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=be348541540b47a8acc55e04bbf6adf1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:24.021158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:24.022633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800101ec-6894-4fba-b4b8-b33b49fff7cc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:24.032551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 270/541] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:23 2022] GET /identity/v3/role_assignments?user.id=be348541540b47a8acc55e04bbf6adf1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:24.043439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6fec8-3baa-4209-b3c9-469e4c002ae6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:24.044444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6fec8-3baa-4209-b3c9-469e4c002ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:24.044926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6fec8-3baa-4209-b3c9-469e4c002ae6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:24.047348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 272/542] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:24.052898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:24.053168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:24.053373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:24.082609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:24.083876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:24.174803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 271/543] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:24.565587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6fec8-3baa-4209-b3c9-469e4c002ae6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:24.608949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:24.698787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6fec8-3baa-4209-b3c9-469e4c002ae6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:24.745115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:24.745752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:24.746035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:24.746306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:24.756995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:24.762230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:24.763118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:24.763944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 273/544] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:24.772459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:24.822306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:24.931469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d162-856c-458f-96c7-898b01be80f1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:24.975152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:24.975793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:24.976079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:24.976353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:24.977037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:24.977342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:24.978722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:24.988086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 272/545] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:24.993807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:25.128267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:25.132108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dac2f-357e-47af-9bd9-66a6b3cd10f3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:25.180872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:25.181532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.181814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.182086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.199343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:25.200825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:25.221184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:25.223333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:25.225459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:25.225459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:25.225459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:25.225459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 274/546] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:25.228016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:25.340902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:25.367326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb18ac-27d3-478e-a45a-b13acb4e78d9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:25.412770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:25.413706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.414189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.414679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.415563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:25.416070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:25.417651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6338f-f5a6-46b3-9b13-1947e151abca None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:25.434405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 273/547] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:25.440347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:25.614140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58b60-f22e-483b-b3c8-7f84ef473f09 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.614507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58b60-f22e-483b-b3c8-7f84ef473f09 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.614686 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58b60-f22e-483b-b3c8-7f84ef473f09 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.616521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 274/548] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:25.621556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efa6f-261c-4007-8437-cb1559a398b7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.621834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efa6f-261c-4007-8437-cb1559a398b7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.622106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044efa6f-261c-4007-8437-cb1559a398b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.650069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-044efa6f-261c-4007-8437-cb1559a398b7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:25.651280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-044efa6f-261c-4007-8437-cb1559a398b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:25.657084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c565d5-534c-48a5-9d30-ce0299dc6354 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:25.721711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:25.722260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.722539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.722815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.742271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:25.743781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:25.773166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:25.774716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:25.776442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 275/549] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:25.781129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:25.788193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 275/550] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:25.802042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:25.888643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833a75-7532-4e8a-b1b9-70f7dda033f0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.888999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833a75-7532-4e8a-b1b9-70f7dda033f0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.889176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833a75-7532-4e8a-b1b9-70f7dda033f0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.899258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 276/551] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:25.900704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:25.900995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:25.901274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:25.938667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:25.939950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:25.973493 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d59154-a559-4d6a-9914-fa42f5699880 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.033177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.033834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.034320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.034510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.035251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:26.035754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.037179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999946f-fc37-4541-ae45-39076f10fa6c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.058860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 276/552] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:26.059278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 277/553] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:26.060195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12a52-7c76-4d47-a900-5ae19f4b52d4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.060483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12a52-7c76-4d47-a900-5ae19f4b52d4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.060762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12a52-7c76-4d47-a900-5ae19f4b52d4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.063034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 277/554] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:26.065418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.068539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.068814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.069080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.075556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.091420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:26.092811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:26.183416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.198899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 278/555] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:26.199817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.224582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593803ec-6340-4836-9fe1-f8fdff4a12d1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.274514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.275115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.275390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.275788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.355468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.357230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.361373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.384158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a405b-2c00-41ac-9b1d-b5fc312370a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.435042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.436938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.437602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.438185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.450717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:26.456640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:26.457743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.458741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:26.461156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 279/556] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:26.462073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.492374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 278/557] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 21:12:26.493102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.541151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.594050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb682-90fc-4e61-9838-aef7abd5a6b1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.602576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12054def-7cca-423f-8e3a-3a3396393ac5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.605840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.634364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.635004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.635277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.635572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.642294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.642894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.643173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.643446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.644137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:26.644437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.645796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.646906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:26.651768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:26.653127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 280/558] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:26.654281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.658234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 279/559] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:26.659034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.758025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24feb2a1-ea47-4570-831e-65490d037d54 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.761155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced5f8-b23a-4d91-a7f1-618d372bfbcf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.777522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.791004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.791657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.791937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.792210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.797098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.797718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.797996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.798268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.803030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:26.808447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-aabc05e8-8030-41d8-b190-f357235b49cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:26.808923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:26.809990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:26.810837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:26.811627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.811699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 280/560] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:26.811699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5489b8b-ea2f-414f-b374-31064aa4f47b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.812280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5489b8b-ea2f-414f-b374-31064aa4f47b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.812407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5489b8b-ea2f-414f-b374-31064aa4f47b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.813709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 281/561] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:26.816309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5489b8b-ea2f-414f-b374-31064aa4f47b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:26.826655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:26.827901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:26.829109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:26.829109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:26.829109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:26.829109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 281/562] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:26.829109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:26.847840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:26.926052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5489b8b-ea2f-414f-b374-31064aa4f47b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.929170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9ff85-f8ce-48d0-9063-b88593d90b17 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:26.966412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.967063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.967871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.968309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.970064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:26.971688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:26.971970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:26.972238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:26.973155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:26.973455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.976635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.984248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:26.985868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:26.988406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 282/563] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:26.989229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5932dba0-d19c-40bb-8093-18d423c61416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:27.006952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.008084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:27.008988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:27.008988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.008988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:27.008988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 282/564] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.009965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.009965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.009965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.034740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:27.036166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:27.053049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.101245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.106305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 283/565] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:27.107147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.114505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423bd5-164f-4447-914d-24eedd1ba2c1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.153955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.154557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.155088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.155570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.156152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.156457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.157958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.171752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 283/566] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.171752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.222105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d924-7c5d-4460-8af8-9b0613bc27db None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.258469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.258998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.259274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.259563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.260200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.260754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.261861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.278303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 284/567] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.278303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.281964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549231ff-4e50-43c1-968e-b48be513f29b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.289412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.318774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789440-342a-4312-86f6-903c64617934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.319396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789440-342a-4312-86f6-903c64617934 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.319952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789440-342a-4312-86f6-903c64617934 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.319952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789440-342a-4312-86f6-903c64617934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.330489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789440-342a-4312-86f6-903c64617934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.331957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789440-342a-4312-86f6-903c64617934 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6d789440-342a-4312-86f6-903c64617934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:27.346043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.347182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:27.348461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:27.348461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.348461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:27.351616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 284/568] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.352400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789440-342a-4312-86f6-903c64617934 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.355128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.387736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcda02-7791-43c8-94ee-d07aa3ffc42d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.421953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.422556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.422834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.423098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.423762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.424061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.425425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.437258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 285/569] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.438050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.458583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789440-342a-4312-86f6-903c64617934 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.491825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.492465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.492744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.493014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.503271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.504761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.504884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:27.519108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.520105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:27.521004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:27.521004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:27.521004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:27.521004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 285/570] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.522324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.538306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45348425-cbb9-441c-81e3-7e3f2ae438e4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.572965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.573581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.573853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.574120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.579517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.581654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.583260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:27.594436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.595349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 286/571] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.596275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.628032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90c156-b39f-4187-9068-17b23df43c9c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.663136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.663763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.664042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.664316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.664957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.665260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.666651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.680713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 286/572] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.682099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.701338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591138f2-fdc5-46f9-95ad-1464fbd46127 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.729418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.735922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.736553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.736832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.737104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.747247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:27.753120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 287/573] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.755021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.784644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02c1cf-a7e6-471b-ae89-5bba0cff7530 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.798689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.820441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.821056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.821327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.821602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.829085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.830535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:27.842279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:27.843395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 287/574] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:27.844974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.863217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cdab3-a33f-4ff8-84ab-c1cea3312882 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.897897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.898496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.898777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.899051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.899718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.900020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.901387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.913068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 288/575] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.913853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:27.945259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c1c6e-f8ee-4eb9-8ed5-f1aad7551c9c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:27.967042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:27.980145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:27.980746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:27.981025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:27.981299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:27.981986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:27.982290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:27.984288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:27.995999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 288/576] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:27.996798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.024914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c093ab-3db8-471b-ada0-beb8e5b1854c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.029577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.060184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.060729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.061045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.062015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.069030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.070448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:28.082010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:28.082955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 289/577] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:28.083959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.100335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cee0b7-22b4-4060-baaf-0c40bb6101e3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.134576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.135172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.135455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.135758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.136411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.136714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.138091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.146054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 289/578] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.146842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.189402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f559c-c47b-4b85-afb7-a47f085f4b01 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.200418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.224885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.225500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.225741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.226005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.226724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.227021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.228448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.242170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.246025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 290/579] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.246832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.248466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd337203-30c8-42a9-9f99-c00915d63d5e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.285984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.286610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.286891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.287164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.296062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.297685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:28.308719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:28.309833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 290/580] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:28.311154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.350271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f0385-d2b9-470c-a8f6-78279d5cd243 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.383274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.383896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.384250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.384520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.385236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=103b5c20d7354245a2c57adb0363ffce) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.385542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.386953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.392333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 291/581] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/role_assignments?user.id=103b5c20d7354245a2c57adb0363ffce&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.392888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.421169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb9b13-0760-4217-b3cd-6c4b277694af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.428806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.459255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.459892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.460167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.460431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.461168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.461466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.462958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.492557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 291/582] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.492557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.509322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.524395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9c72-49c2-42fe-bf43-d17659d0d8ba None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.567040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.567621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.568081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.568365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.581204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.582669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:28.598427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:28.599422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:28.600349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:28.600349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.600349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:28.600349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 292/583] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:28.600349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.609300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f8bc-df62-42aa-8867-366bd606c67b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.647893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.648529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.648809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.649080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.660779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.662237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:28.677766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:28.678757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:28.679717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:28.679717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.679717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:28.679717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 292/584] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:28.679717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.706611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.726640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470473b-d179-4689-96a8-8c1fafbc9bac None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.753373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.766719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.767115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.767395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.767700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.768478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.768733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.770188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.795901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 293/585] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.796083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea2091-9af0-4aea-a799-05882e4aece6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.796220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.831805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.832340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.832622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.832896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.844052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.845540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:28.860097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:28.861072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:28.862552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:28.862552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:28.862552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:28.862552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 293/586] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:28.862552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.910960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ef6d-4225-4f37-b6d4-a29be9ac25ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.948226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:28.948835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:28.949117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:28.949398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:28.950040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:28.950346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:28.951772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:28.963293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 294/587] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:28.964144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:28.966764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:28.981057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb5feb-2952-4546-96e7-d608ce0b13f8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:28.999794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.018821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.019429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.019744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.020018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.020678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.020981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.022367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.042803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 294/588] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.042803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.080477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24aa5-c6a2-4677-8213-85e5cb0641c4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.116430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.116981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.117257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.117526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.129025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.130451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:29.145425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:29.151323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:29.152979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:29.152979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:29.152979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:29.152979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 295/589] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:29.152979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.157557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcad87-b32b-44dd-b451-c59e6da53aca None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.194370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.195133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.195523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.195889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.196525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.196822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.198285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.208037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.215780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 295/590] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.215780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.254576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.259302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ba29-b2df-418e-99b4-0f8f879a456e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.296611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.297416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.297695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.297968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.305598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.307017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:29.319615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:29.320589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 296/591] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:29.321586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.328703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ca549-3aa3-450a-b07a-c74e0fef40c7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.362721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.363347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.363655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.363930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.370482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.371882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:29.383881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:29.384939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:29.385847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 296/592] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:29.386387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.438245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ab9cf-0ad4-4e9a-977a-a77d09be39d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.454352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.479263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.480030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.480376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.480644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.481279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.481576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.483461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.486203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.490155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337354-491d-48f9-9d4b-22f9f5fa0123 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.500871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 297/593] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.500871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.530184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.531168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.531451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.531905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.538863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.540337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:29.552914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:29.554036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 297/594] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:29.555136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.610303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceba2c-3d0d-4594-b7bb-5644878353b3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.646988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.647852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.648134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.648406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.649137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.649451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.650853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.671575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 298/595] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.671845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.678346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a4131-d0a1-40a8-a529-11242c4e5430 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.703448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.714362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.714970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.715248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.715518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.716356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.716660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.718819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.733436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 298/596] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.734245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.757087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.785917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff44c5-f93d-472c-8e8d-b07309c2d7b9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.823788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.824426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.824707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.824976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.831942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.833361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:29.846009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:29.846986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:29.848045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:29.848510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85170-40a7-4202-88bf-c180fcdddfab None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.852012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 299/597] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:29.852681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.883005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:29.883725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:29.884067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:29.884424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:29.885109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:29.885413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:29.886897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:29.898903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 299/598] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:29.899964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:29.954317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:29.964320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20e3eb-723f-453e-9ea0-65c2f3c3db40 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:29.979245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.002910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d90fc-3389-4c53-b824-465392fb685b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.005026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.006513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.006793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.007063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.020890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.022322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.038123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.038785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:30.038889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:30.039775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:30.040665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:30.040665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:30.040665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:30.040833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.040833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8985883a056c431b885bb07303d3f8af/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.040972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 300/599] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:30.040972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.059524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=8985883a056c431b885bb07303d3f8af, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.061101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.076249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 300/600] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:29 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8985883a056c431b885bb07303d3f8af/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:30.077936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.163131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfb6ef-9e7b-4232-a38c-981a287e243c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.209908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.210298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.210579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.210850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.211641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:30.211951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.213502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.228586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 301/601] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:30.229122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7b8d8-9b78-4f4c-9dc7-ede0bc3ae631 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.234828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.240607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.272883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.285902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.285902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.285902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.286412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.306521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=dac985d2461d4132af309ded8c7ce1a1, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.308014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.325306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 301/602] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:30 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:30.326002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.373256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43326548-7321-4664-89b1-b3dc95bc1a56 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.412943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.413681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.413911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.414185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.414890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:30.415206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.417196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.436401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 302/603] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:30.438960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.531551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a66d-4ffa-42a6-8e3f-c4ef8b246683 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.531924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.605438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50043b72-4565-4bd6-bfc0-290311713b23 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.615647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.616791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.617236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.617655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/b670a7afd2264609894dec2feabc4581/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.635515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=b670a7afd2264609894dec2feabc4581, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.637234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad65b72-b0a2-4329-b687-f6e0e947e653 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.656273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 302/604] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:30 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/b670a7afd2264609894dec2feabc4581/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:30.657527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2797a-4704-4fdf-999a-9c72e19b6caa None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.657810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2797a-4704-4fdf-999a-9c72e19b6caa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.658128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2797a-4704-4fdf-999a-9c72e19b6caa None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.659279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.659625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 303/605] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:30.660310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.660752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.661165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.665406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.665591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.665914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.670333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.672782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.687114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:30.688643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:30.689711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:30.690831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 303/606] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:30.691504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:30.697121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:30.726120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.795314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 304/607] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:30.884625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641438e-a3ea-4bc6-8950-a91b865cf8a8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:30.914179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:30.926795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:30.927334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:30.927645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:30.927919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:30.928612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:30.928917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:30.930345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:30.947647 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 304/608] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:30.950283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:31.074975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2a50e-cac0-41cf-96b3-f72d200586d0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:31.116124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:31.116786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:31.117066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:31.117342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/be348541540b47a8acc55e04bbf6adf1/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:31.133397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=be348541540b47a8acc55e04bbf6adf1, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:31.134862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:31.156490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:31.157275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 305/609] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:30 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/be348541540b47a8acc55e04bbf6adf1/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:31.312954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7ecc-6bed-4e9e-956b-3baa96d87fad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:31.365253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:31.366292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:31.366793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:31.367285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:31.388552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:31.394832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:31.395817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:31.396860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:31.400229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 305/610] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:31.405800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:31.417871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:31.558991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e1994-9865-452d-ba8a-31a5c230cf2f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:31.601844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:31.602384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:31.602666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:31.602941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:31.603665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:31.603972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:31.605384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:31.616543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 306/611] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:31.622080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:31.677919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:31.742066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b88bf-b92c-4180-88e7-aa14232a4c48 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:31.783622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:31.784256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:31.784541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:31.784815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:31.795348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:31.796773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:31.815294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:31.816347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:31.817252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:31.817252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:31.817252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:31.820543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 306/612] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:31.826221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:31.932025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:31.980128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402987f1-44d1-4ec8-a03c-716e8029d18e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:32.031339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:32.031995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.032279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.032555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.033214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:32.033521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:32.034913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:32.052646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 307/613] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:32.056602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:32.220002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:32.225754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4c689-fdc0-4ce2-90a2-87208df63ed8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:32.284800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:32.285444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.285725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.285996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.300264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:32.301717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:32.314120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:32.315609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:32.317825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:32.320027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 307/614] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:32.321301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:32.481868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cc03-4fde-43ee-bdb1-2bc47a173cd9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:32.525071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:32.534160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:32.534803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.535057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.535332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.536153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:32.536472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:32.537927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806d320-da56-4bfc-ae2a-91f3539684c5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:32.563156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 308/615] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:32.568602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:32.620051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6404-4dc8-4ef4-9869-98af01b6686f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.620504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6404-4dc8-4ef4-9869-98af01b6686f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.620785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6404-4dc8-4ef4-9869-98af01b6686f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.622873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 309/616] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:32.628936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.630848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.630848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.665069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:32.666254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:32.752751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639a836-3d44-4c58-82a3-a838a565bfdb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:32.793486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 310/617] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:32.794758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:32.796251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:32.796646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:32.796919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:32.814162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:32.816104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:32.836819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:32.837838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:32.838733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:32.838733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:32.838733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:32.840527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 308/618] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:32.846796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:32.987072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4cafd-4d68-469f-9a3b-0c668782105b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.005345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.024487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.025100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.025374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.025647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.026288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:33.026639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.028301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e72c7-1f5f-4321-b9b3-ab1a8ed62809 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.040795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 311/619] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:33.046133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.066411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.082964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c7a07-e870-4f00-afb2-e4df17347732 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.082964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c7a07-e870-4f00-afb2-e4df17347732 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.083208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c7a07-e870-4f00-afb2-e4df17347732 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.084596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 312/620] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:33.090086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.090367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.090641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.112146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:33.113342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:33.189759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 313/621] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:33.190620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.198174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f426b30-84c4-49fa-b5f1-c974920dcca5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.247271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.247942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.248281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.249172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.261654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.263029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:33.283842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:33.284785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:33.285680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:33.289505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 309/622] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:33.295896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.358285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcac54-57cb-47e5-81f3-9a3e179c83a2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.370247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.379889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.409903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda53bd-0a86-4c9e-8d70-937c1b95577b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.448124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.448736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.449018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.449299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.449991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:33.450299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.452330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.456229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.456859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.457136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.457468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.476600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:33.487990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 310/623] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:33.490408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:33.492548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:33.494516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 314/624] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:33.499552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.515106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.622318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3289f-1956-4827-b7aa-0433aba56d60 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.638428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fe6e-5e1b-414d-9e8b-41fb6b556989 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.658274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.666944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.667731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.668014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.668594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.669556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:33.669858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.671481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.676993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.677662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.677937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.678215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/103b5c20d7354245a2c57adb0363ffce/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.683946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 311/625] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:33.683946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.699091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=103b5c20d7354245a2c57adb0363ffce, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.700901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52901a22-c05e-4e97-bc38-7fa092d2d8e9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.718226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 315/626] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:33 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/103b5c20d7354245a2c57adb0363ffce/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:33.719605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501dba6-c997-42da-a9c7-8314fc105d43 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.719976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501dba6-c997-42da-a9c7-8314fc105d43 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.720354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501dba6-c997-42da-a9c7-8314fc105d43 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.721854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 316/627] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:33.722570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501dba6-c997-42da-a9c7-8314fc105d43 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.748940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.815068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cf1a-5a40-42b0-8d01-4222216046fa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.855094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.855890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.856234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.856581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.867208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501dba6-c997-42da-a9c7-8314fc105d43 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:33.869317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ffbcb56c-c785-43b7-8643-e683da9315d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:33.875506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:33.876543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 312/628] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:33.882179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.883101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.883101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.907117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:33.907117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:33.923413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:33.924143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:33.924399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:33.924664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:33.941381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:33.942946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:33.962951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:33.964050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:33.964961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:33.964961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:33.964961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:33.965143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 317/629] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:33.965950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:33.966089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:33.992453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 313/630] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:33.994319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.066870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.115200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445a625-9ee1-41cf-959f-cdfb26e8adc2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.122610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3912d6b-2511-41e8-96ed-a4aebd87f355 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.164109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.164722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.165258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.165542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.166191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:34.166490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.167868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192ac69-70c9-441a-a36f-ebae4d0639a8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.180725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.181335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.181615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.181795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 314/631] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:34.181891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.182545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:34.182846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.183018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abdad7-bea5-496b-ab65-cf9a63b310dd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.183271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abdad7-bea5-496b-ab65-cf9a63b310dd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.183559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abdad7-bea5-496b-ab65-cf9a63b310dd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.184253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c9a2a-7241-40da-9135-b5b37e3e23bc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.188944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 315/632] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:34.189702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abdad7-bea5-496b-ab65-cf9a63b310dd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.198953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 318/633] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:34.200227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.200510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.200785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.224364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:34.225534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:34.239983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.305998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abdad7-bea5-496b-ab65-cf9a63b310dd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.323842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 319/634] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:34.324673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.349006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.349637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.349969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.350244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.358999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.360503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:34.372210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:34.375016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 316/635] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:34.377384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.408205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.462205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f01fc-da61-4729-97c0-4834d35f4e8a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.487150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.490489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfedb-0335-4a4a-8a48-65220ca37939 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.509779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.510633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.511055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.511501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.526609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.527330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.528025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.528306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.528572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.529247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.539933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:34.546039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:34.546985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:34.547897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:34.548553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:34.549436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:34.550340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:34.550340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:34.550340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:34.552586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 317/636] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:34.553602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 320/637] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:34.554166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.554521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.669426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9fcac-a778-4c21-8955-073f9a7ef551 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.677891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3deb-ccc8-4f16-a708-97ac701d44e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.692872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.704248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.704702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.704980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.705270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.705966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:34.706267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.707707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.714835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.715439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.715751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.716024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.716667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:34.717584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.718966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.730481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 318/638] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:34.730481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.737426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 321/639] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:34.742256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.756233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:34.848059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b2d42-d0cd-4d4c-b5aa-a9921ab4bb81 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.849621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46a016-a60e-40a6-b56d-a891bde42a03 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:34.884088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.884710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.884989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.885262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.890692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:34.891395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:34.891720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:34.891993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:34.892660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:34.892965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:34.894323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:34.896728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:34.901474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:34.902626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:34.903468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:34.907700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 322/641] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:34.907821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 319/641] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:34.908432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.908539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:34.955682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.003107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.024757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad426eba-a9e5-4f6d-9ef1-42e20b685630 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.029582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5164-61dc-4deb-b15b-b9f217cdb717 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.058881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.059508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.059814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.060089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.065694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.066293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.066571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.066838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.068396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.069821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:35.081538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:35.082575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 320/642] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:35.084109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.144750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.146159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.178699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.193077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf4580-93b2-4e34-b916-28d1a9b17487 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.232096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.233642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.234095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.234565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.244936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.247589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 323/643] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:35.248103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.253157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.254962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:35.270863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:35.271844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:35.273447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:35.273447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.273447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:35.273447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 321/644] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:35.273447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.353362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaa2f0-b830-4fa1-8595-6695945c3342 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.384529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c351ca-bcbc-4a9b-9c66-a921d677a45e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.386247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.386851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.387133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.387391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.388089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:35.388388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.390313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.391682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.399310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 324/645] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:35.400078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.428850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.430124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.430818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.431267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.432197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:35.432675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.434277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.456155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 322/646] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:35.457275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.512728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cf795-f0af-4c5e-946b-4092a39f6bad None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.513848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.548485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.549108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.549379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.549645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.561245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.563200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.572921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5849f4-a130-4433-910b-f24cc90e489a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:35.577765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:35.578723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:35.584244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:35.584244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.584244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:35.584244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 325/647] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:35.584244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.620043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.620658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.620938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.621213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.621863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:35.622169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.626975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.637238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.646750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 323/648] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:35.647861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.696691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215462a-0c6c-47e4-a685-316ac20a4159 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.737044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.737982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.738271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.738559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.749955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.751408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.763007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf715-5bfe-43f2-ab1b-aeaa4456b2e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:35.767057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:35.768050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:35.774277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:35.774277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:35.774277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:35.774277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 326/649] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:35.774277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:35.777299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.805480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.806081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.806361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.806632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.866374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:35.869545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06380d-252b-4875-aec9-6f9ba4d8213e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:35.891873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.894709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.903003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:35.903650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:35.903934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:35.904211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:35.905409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:35.905713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:35.907090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:35.919151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 327/650] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:35.920683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.021864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 324/651] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:36.022632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.032431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c703df-9f55-4d12-a6cf-f2d631155534 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.037746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.069266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.069888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.070161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.070428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.078043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.079505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:36.090795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:36.092692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 328/652] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:36.093816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.114841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.128316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e5193-455e-45a8-b414-b519d9922b05 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.162310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.162910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.163189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.163459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.164122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:36.164420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.165767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.177204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 325/653] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:36.177994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.196739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df968b26-80e5-4648-a2be-7b8b4d2a5105 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.230871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.231627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.231905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.232296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.238680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.240164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.246342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9886654a-6b97-441f-81e5-49fe52acb49b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:36.251471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:36.252444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:36.253347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 329/654] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:36.253835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6a94-2136-4060-9931-9941af3c56c6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.253964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6a94-2136-4060-9931-9941af3c56c6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.254248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6a94-2136-4060-9931-9941af3c56c6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.255588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 330/655] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:36.256097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6a94-2136-4060-9931-9941af3c56c6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.282689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df700f-e65b-4dd4-b054-c6141906d126 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.314908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.315551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.315841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.316113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.327705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.329157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.338466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:36.344027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:36.344997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:36.345913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:36.345913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:36.345913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:36.345913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 326/656] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:36.345913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.364956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6a94-2136-4060-9931-9941af3c56c6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.397992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.398582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.398859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.399122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.399868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:36.400166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.401624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.418466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 331/657] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:36.419244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.447930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4b313-1534-47ac-86bd-5973d6fda023 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.470523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.482285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.483002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.483281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.483573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.490895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.492363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-89d90cd4-d8a2-4b73-86a6-256698592b8b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:36.503941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:36.504915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:36.505893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 327/658] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:36.506663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.506663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.506663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.522994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b7de-36e1-4bfe-a03b-23fea899dc37 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.524160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:36.525345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:36.558154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.558687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.558968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.559235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.559955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:36.560259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.563092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.571437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.580334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 332/659] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:36.581249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.597657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 328/660] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:36.598507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.690038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565cd77-3fc2-4837-8d5d-1a0ca00e9539 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.715603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6fe13-04ad-46ba-915e-e42051304fd8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.724693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.725407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.725786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.726165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.726908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:36.727212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.728611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.731340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.740691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 333/661] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:36.745335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.760479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.761095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.761376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.761754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.807685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:36.836733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.838264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.852315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cc4ab-ffac-4b73-b6d4-e53f5483ec2d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:36.887905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:36.888507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:36.888827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:36.889100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:36.889934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:36.890238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:36.891696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:36.909688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 334/662] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:36.910530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.942998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 329/663] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:36.943877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:36.960195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.012594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d1cbf-9e2e-45d3-ad4c-2e392f4edb4c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.028700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.042656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d106e-0a8e-479e-9efa-f84ae0e22062 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.046702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.047304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.047618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.047898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.048628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8985883a056c431b885bb07303d3f8af) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:37.048965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.050413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.056082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 335/664] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/role_assignments?user.id=8985883a056c431b885bb07303d3f8af&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:37.056865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.078649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.079427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.079738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.080010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.087970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.089349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:37.101692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:37.102672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:37.103595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 330/665] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:37.104088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.189184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c46c-24b0-41b4-b2af-9dbdc8515197 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.190143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.225175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076683a-a1fe-4209-b650-38763dec4bd6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.227330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.228007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.228287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.228562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.242487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:37.247842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:37.249042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 336/666] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:37.249901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.268293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.268911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.269915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.269915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.299400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.348952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.350431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.360450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848808-910c-42ea-a52f-98e05a072e79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.396163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.396943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.397225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.397496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.408944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.410393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:37.424800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:37.425782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:37.426735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:37.426735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:37.426735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:37.427845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 337/667] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:37.429462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.432236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.466040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 331/668] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:37.467260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.531145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.541455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6003c9-0030-464b-8e8c-91c548ceef91 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.578956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.579295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6618c8-b0e8-4e56-a385-a496bde3fab9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.579590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.579873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.580144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.580840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:37.581147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.582594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.595127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 338/669] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:37.595991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.620433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.620960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.621239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.621512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.622178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:37.622482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.624663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.639716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 332/670] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:37.641413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.695298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.735103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934aea2-2786-4e32-af9d-493d2f82e0c4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.776096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.777238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.777517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.777789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.778441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:37.778740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.783691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4899e-ec6e-46c0-839c-66972ed926d3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.784688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.810786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 339/671] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:37.812280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.829009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.829643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.829908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.830169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.830726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.844027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:37.845622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:37.863111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:37.864189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:37.865163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:37.865163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:37.865163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:37.867685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 333/672] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:37.868707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:37.949384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf89ed-4506-4372-84be-e1c6c82510d6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:37.981730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:37.995349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:37.996407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:37.997293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:37.997567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:37.999041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:37.999346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.001512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c87dd-3366-40cb-9796-5f80fc3a3621 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.005181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.019855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 340/673] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:38.020855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.080719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.083224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.084232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.085107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.110865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.114112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.115123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:38.135717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:38.136748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:38.137656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:38.137656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:38.137656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:38.145260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 334/674] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:38.145260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.155301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2c13a-ffd4-4e0b-877c-b55f2e461de6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.225785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.226641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.226925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.227196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.237295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.240932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:38.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:38.261587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 341/675] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:38.262491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.305649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ea61-3bc1-48d3-b7a9-213b6565df56 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.356731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.358349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.358620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.358978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.359741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:38.359980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.361604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.375359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 335/676] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:38.375728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:38.380733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.398673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac7350-0104-458d-85c8-0f597a2d005d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.447809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.448440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.448715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.448986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.449623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:38.449922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.451902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.456993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:38.474798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 342/677] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:38.474798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.512838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad6509-cc29-4026-8d51-8fb1149d4966 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.554004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.554641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.555101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.555379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.556430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:38.557037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.558470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.578626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 336/678] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:38.579548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.607680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa771e2-b3b1-413e-8343-271a653c2345 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.652607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:38.656478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.657105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.657382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.657651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.671463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.673291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:38.690065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:38.690976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:38.691918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:38.697534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 343/679] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:38.698652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.711515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5e810-4314-4bbd-a028-9700c6596d94 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.753850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.754483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.754757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.755024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.766689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.768096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.774776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:38.780982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:38.781933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:38.782948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:38.786907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 337/680] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:38.787812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.835861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166d58d-a5dc-4578-92ed-5833cd2d8b54 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.874052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:38.875010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:38.875508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:38.876040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:38.876963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=b670a7afd2264609894dec2feabc4581) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:38.877480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:38.879124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:38.886301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 344/681] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/role_assignments?user.id=b670a7afd2264609894dec2feabc4581&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:38.887587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:38.946810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e59bb-c264-4fdc-b0f6-cc5a1f8c02c0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:38.993997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:39.037014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.038200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.039820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.041060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.043114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:39.044330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.048052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.075245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 338/682] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:39.084564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.129561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393615e6-0443-4ce3-a910-0dedec5d21ba None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:39.169332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:39.188581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.189009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.189297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.189574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.190309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:39.190615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.196690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.218724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 345/683] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:39.219865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2e641-23a5-48be-8c2a-261ed68c33c0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:39.224963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.263321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.264886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.266312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.266830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.267829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=be348541540b47a8acc55e04bbf6adf1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:39.268354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.270077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98290435-3e35-4d7c-b3f1-577600ce1d18 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.279214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 339/684] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/role_assignments?user.id=be348541540b47a8acc55e04bbf6adf1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:39.349687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3faba-e8fc-408d-9e20-2b019fcbf345 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:39.388919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.389531 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.389814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.390090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.403281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45260f8d-5024-42ab-8f43-09088fa23d77 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.403929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45260f8d-5024-42ab-8f43-09088fa23d77 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.404552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45260f8d-5024-42ab-8f43-09088fa23d77 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.408425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 340/685] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:39.412171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.412447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.412716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.448056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:39.449978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:39.451596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:39.475210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.477166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.523404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:39.585502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 341/686] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:39.609339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 346/687] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:39.614549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.740253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac234-9ea1-44b0-8855-62ff20153a47 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:39.784230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:39.784868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:39.785150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:39.785427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:39.803067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:39.804595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:39.824550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:39.825507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:39.826807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:39.826807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:39.826807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:39.826807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 342/688] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:39.831398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.873899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:39.903830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:39.980865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:40.024142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad28c2-a67b-4475-822f-f96852ac335f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.053088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1969e-cfca-47ca-9410-c796222ad939 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.091421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.093162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.093703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.094187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.102702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.103332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.103635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.103912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.104575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:40.104874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.106372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.108173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.109897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:40.128195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:40.129202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:40.130119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:40.130119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:40.130119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:40.130119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 343/689] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:40.135684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 347/690] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:40.136021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.141829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.271924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:40.318466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50c4d-90bc-4eb2-9288-7964504117bd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.346275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd81cf-9e95-409d-b4d2-47814c0024fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.390358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.391035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.391318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.391607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.398823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.400473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.402413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:40.408257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.409151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.409588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.410022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.410824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:40.411278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.412824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:40.415130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:40.416287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:40.417406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:40.423556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 348/691] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:40.431312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.437859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 344/692] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:40.443278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.604638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe0721-e2c8-4b23-b473-61c346001c0f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.644923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.646121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.646552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.646975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.647858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:40.648311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.650299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e59b3-fdb0-446e-8f78-2d3758d67fe0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.660354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104694c-cb81-4f03-a505-6855728645ac None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.665892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 349/693] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:40.671478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eafb6-e42c-4031-85a4-0d5709c6a421 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.671823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eafb6-e42c-4031-85a4-0d5709c6a421 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.672103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eafb6-e42c-4031-85a4-0d5709c6a421 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.674160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 350/694] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:40.674886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eafb6-e42c-4031-85a4-0d5709c6a421 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:40.706953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.707878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.708406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.708824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.709675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.712102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c877f9-a767-476f-bfff-e3aae231d939 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.725593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:40.743979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 345/695] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:12:40 2022] POST /identity/v3/users => generated 254 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:40.745765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.746387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.746820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.762449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:40.772194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:40.773725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:40.879376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eafb6-e42c-4031-85a4-0d5709c6a421 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:40.942914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 346/696] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:40.950443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:40.951448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:40.952035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:40.952458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:40.953338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=103b5c20d7354245a2c57adb0363ffce) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:40.953825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:40.955524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:40.962283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 351/697] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:40 2022] GET /identity/v3/role_assignments?user.id=103b5c20d7354245a2c57adb0363ffce&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:41.224661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:41.365318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc0145-c4df-4b50-a65d-c500ee5e8af9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:41.393948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:41.406900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:41.408172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.408624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.409002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.420788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:41.425710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:41.426651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:41.427510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 347/698] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:41.434698 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:41.573542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17f3dc-1757-4782-8f5e-867060dec1f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:41.607509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:41.629094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:41.629776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.630064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.630342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.631035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:41.636213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:41.637651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c13a8-ca30-4f85-bf12-4deef0aa53bf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:41.656240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 352/699] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:41.662267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:41.752740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b324e-bc92-4cad-9041-fa2cf7971d0d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.753064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b324e-bc92-4cad-9041-fa2cf7971d0d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.753306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b324e-bc92-4cad-9041-fa2cf7971d0d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.755357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 353/700] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:41.759603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6d3d-693c-4e7e-b1ac-b992af4017e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.760127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6d3d-693c-4e7e-b1ac-b992af4017e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.760606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d6d3d-693c-4e7e-b1ac-b992af4017e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.787419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fd8d6d3d-693c-4e7e-b1ac-b992af4017e3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:41.793524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fd8d6d3d-693c-4e7e-b1ac-b992af4017e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:41.826291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e667a-b03b-448c-b4f0-cd4fa8f9a30e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:41.877992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:41.878651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.878934 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.879207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.894907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:41.896401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:41.921972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:41.923028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:41.923991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:41.923991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:41.923991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:41.924811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 348/701] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:41.936703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:41.943132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 354/702] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:41.987331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f34b-d077-4eba-a92e-8cafd1fc9ac3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.988031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f34b-d077-4eba-a92e-8cafd1fc9ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.988530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f34b-d077-4eba-a92e-8cafd1fc9ac3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:41.990645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 355/703] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:41.996292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:41.997030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:41.998159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.014743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.037461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:42.038671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:42.145400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a51891-65aa-44c1-97ca-84e731bbef8d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.174487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 356/704] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:42.189779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.190409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.190697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.190974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.191658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:42.191965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.193343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.217505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 349/705] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:42.222877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.245916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.319876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.374491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.381006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098d7a8-9bce-450d-b13b-20d9273d58f4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.423310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.423975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.424254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.424527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.425110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.426476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.439847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a57fbd-f066-4ab9-954d-be3593f80caf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:42.452909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:42.453962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:42.455192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 350/706] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:42.467929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.516634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.517810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.518532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.519022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.529509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.531826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:42.549662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:42.550681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:42.551676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:42.552921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 357/707] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:42.554379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.606954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e247-024a-4eb3-8ab8-e42349db77d4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.650611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.651221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.651497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.652062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.652795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:42.653100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.654604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.664654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 351/708] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:42.664654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.680309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103dc1-7520-4b7d-8b99-b2c58064b8c5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.682910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.695056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.728609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.729789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.730114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.730385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.731235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.733693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:42.750037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:42.751081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:42.751967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 358/709] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:42.752814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.787724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb45eb-f777-4281-8f5a-466e3e61731b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.827912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.828764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.829194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.829683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.830734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:42.831296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.833699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.854176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 352/710] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:42.855316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.896333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be114c7-f3f9-4e57-965e-737867ca2512 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:42.942194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:42.942728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:42.943056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:42.943373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:42.944064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:42.944413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:42.945791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:42.953617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:42.955691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 359/711] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:42.956801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:42.998815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510a03-fba4-48ba-ac3e-88affdd4be5d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:43.022770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:43.086842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:43.087470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.087796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.088065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.088706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] RBAC: Adding query filter params (name=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:43.089005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:43.090391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:43.103656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 353/712] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:43.114539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:43.188926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fbf91-14cb-4467-9c4c-7f627b2d357e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:43.231494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:43.232997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.233284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.233621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.243824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652248c-c54a-4e67-8e88-a9442ad55b8a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:43.284612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:43.285373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.285539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.285813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.286515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:43.286812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:43.288270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:43.295939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 354/713] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:43.326638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:43.328508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21330cf5-8e2f-47a1-8d3f-090ed872cbbd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:43.364777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:43.458847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 360/714] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:43.464682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:43.486741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:43.616706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3cf0-52d3-4b16-ab93-1666b9dc2974 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:43.670252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:43.670513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273fa4-6a81-48e5-bb92-0079802424ca None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.670611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.670689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273fa4-6a81-48e5-bb92-0079802424ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.670761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.670842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273fa4-6a81-48e5-bb92-0079802424ca None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.670913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.671476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 361/715] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:43.675164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b99d5-db72-4a1a-857f-35a659cc4d10 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.675649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b99d5-db72-4a1a-857f-35a659cc4d10 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.676110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b99d5-db72-4a1a-857f-35a659cc4d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.689846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:43.691579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:43.713778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f57b99d5-db72-4a1a-857f-35a659cc4d10 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:43.715867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f57b99d5-db72-4a1a-857f-35a659cc4d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:43.721576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:43.723183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:43.724164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:43.724164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:43.724164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:43.724164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 355/716] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:43.729435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:43.802189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:43.807977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 362/717] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:43.810691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1b8e-c60d-4a5d-b0b2-84176a8eb941 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.811248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1b8e-c60d-4a5d-b0b2-84176a8eb941 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.811705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1b8e-c60d-4a5d-b0b2-84176a8eb941 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.817376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 363/718] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:43.819355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.819815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.820324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.863048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:43.866063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:43.867932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a9723-ced6-423e-9cb6-e2a8b32bc8ce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:43.943472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:43.944681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:43.947941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:43.948228 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:43.948906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:43.949213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:43.955123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:43.972637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 356/719] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:43.984508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:43.985990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.005038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 364/720] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 21:12:44.083864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.102937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a2f9f-f8e9-492a-84f5-1ad839e8a454 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.126002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:44.161002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.161645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.161917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.162184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.173174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.174610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:44.202385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:44.203367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:44.204319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:44.208225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 357/721] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:44.210427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.253607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82204-1df1-48ec-86b9-b25bbcba5c48 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.299032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.299690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.299968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.300238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.319994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:44.327031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:44.328009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:44.328846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:44.329495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 365/722] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:44.333736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.383158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d920-e243-4f86-b1d5-ad8b449b65b2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.398229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:44.434488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.436668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.437474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.437953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.439612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:44.440109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.442418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.457839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05459a-cf6c-4d3f-b03c-6061c035284f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.461408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 358/723] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:44.462626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.498669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.499412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.499775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.500113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.500903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:44.501195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.502740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.512342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 366/724] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:44.512754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.523242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:44.611129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79e80-25ee-45f6-ae11-b951d4f742b7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.636840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e45a-ffb1-47ce-bace-83b8850551fd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.652729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:44.661140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.661784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.662065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.662335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.674347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:44.680550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:44.681592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:44.682452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:44.684049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.685416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.686296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.686826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.687084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 359/725] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:44.687962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:44.688089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.688564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.690178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.698716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 367/726] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:44.700283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.821162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c54134-1a79-41a7-8e89-65ff8da0f881 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.823844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:44.886037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.886698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.887097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.887385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.890933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835bc69-33d8-4cf9-9713-71526061d2c8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:44.910027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.911511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:44.937718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:44.938727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:44.940648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:44.940648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:44.940648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:44.940648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 360/727] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:44.946837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:44.952585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:44.953200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:44.953483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:44.953762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:44.954433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:44.954741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:44.956153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:44.970815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 368/728] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:44.971954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.038992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.133970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543804-f628-4044-8527-abe63dcfef5f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.137616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e184f49-5315-4406-8709-7abda8124e30 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.201350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.202692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.206283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.206803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.207777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.208211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.209162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.209770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.211527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:45.212008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.214608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.230798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.233028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.241372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 361/729] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:45.244880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.264595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:45.265552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:45.266481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:45.267377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:45.267377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:45.267377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:45.273205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 369/730] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:45.274175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.384246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d675de-3d78-488b-ba7f-1716c3415281 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.454945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.456471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.457047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.457423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.457768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.467500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd80403-580a-42c9-8c19-5a03ea4c0708 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.472454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.473946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:45.497290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:45.499002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:45.500016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:45.503707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 362/731] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:45.509042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.511999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.512878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.513915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.514386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.515234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:45.515793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.517461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.535709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 370/732] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:45.537293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.649436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.652656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab133e2-715f-43d8-af48-5821376d1c34 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.683195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1cff4-9b1f-466b-87ee-3ce9cc783998 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.690166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.690923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.691267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.691637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.699568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.702022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-80e7315a-26d6-4653-a8aa-80180c9a82c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:45.714514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:45.715580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:45.716961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:45.720252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 371/733] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:45.722351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd69fc-2a0e-49ed-97d9-f7c49c15dae0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.722724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd69fc-2a0e-49ed-97d9-f7c49c15dae0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.723127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd69fc-2a0e-49ed-97d9-f7c49c15dae0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.725288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 372/734] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:45.726168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd69fc-2a0e-49ed-97d9-f7c49c15dae0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.727978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.728978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.729731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.730472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.731500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:45.732104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.733992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127cf2f-ccb1-45c9-b9b9-6cbb4e60f7d5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.743129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.754703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 363/735] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:45.756378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.756902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.757386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.778810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:45.780337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:45.844526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd69fc-2a0e-49ed-97d9-f7c49c15dae0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.855454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 364/736] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:45.857436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.885002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:45.885618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:45.886060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:45.886284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:45.887028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:45.887330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:45.888888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:45.912094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 373/737] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:45.913601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:45.930110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:45.977123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27a6be-6abc-4725-bb04-aaf92ed3bc76 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:45.991318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.015444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.016453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.016982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.017491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.024597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31799f33-66a3-44bf-8916-cfa94ede3b21 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.059464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.060769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.061045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.061316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.094202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.096232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f505bbd-9d6e-4580-9b40-d9c49b8b63a5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.129723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.131174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.161591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.200248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.208686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 365/738] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:46.210188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a374c-9b96-4691-be50-de4d9ee1208f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.210607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a374c-9b96-4691-be50-de4d9ee1208f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.211021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a374c-9b96-4691-be50-de4d9ee1208f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.212707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 366/739] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:46.213519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a374c-9b96-4691-be50-de4d9ee1208f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.248661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 374/740] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 21:12:46.249435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.329477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a374c-9b96-4691-be50-de4d9ee1208f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.354446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890004a7-cf9a-4a6b-9d25-306028af137e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.364844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.365442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.365716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.365981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.366622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.368019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870adfb-a157-4c3d-87f7-7cfef31ed385 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.388532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 367/741] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:12:46 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:46.390653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.390946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.391222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.414503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:46.415778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:46.416458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.417437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.418056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.418576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.435026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.437783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.439821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:46.460551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:46.461627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:46.462553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:46.462553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:46.462553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:46.464800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 375/742] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:46.466606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.490259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.511657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 368/743] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 21:12:46.511657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.637564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2d4a-18de-49b8-8d08-d045f7ca4059 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.638755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b64ed1-a868-45a3-912b-439d2d90f9c5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.675876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.676516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.676794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.677060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.685011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.685690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.686028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.686303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.686953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:46.687352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.688827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.691359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.693579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.703023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 369/744] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:46.707997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:46.711361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:46.712385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:46.713434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:46.713434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:46.713434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:46.713948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 376/745] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:46.720641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.724191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.812653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:46.851569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4a07c-1bb1-4874-a075-e27a9692f645 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.891002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef226c5-f2d8-4c8f-86e3-8df7d03738aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:46.897530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.899629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.900139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.900617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.902108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:46.903272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.905810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.923341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 377/746] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:46.928538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:46.930472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:46.931229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:46.931510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:46.931824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:46.942825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:46.944627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:46.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:46.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:46.960427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:46.962578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 370/747] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:46.963604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:47.048385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:47.052798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbe342-7a2e-4534-819a-b1e4559dbccc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.088451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234ac5-9109-4afb-badd-5530efd0b74b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.095275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:47.101976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.102591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.103252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.103577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.104236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.106142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.135710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 378/748] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:12:46 2022] POST /identity/v3/services => generated 240 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:47.137126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:47.152995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.155087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.158619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.159231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.175170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.177261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:47.201361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:47.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:47.203304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:47.208707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 371/749] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:47.214750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:47.325360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e36cc1-5d53-4eb3-abb0-86ebc64f8072 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.351496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfd767-54ea-47f8-a4c6-bdcbca02c984 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.366743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.367502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.367963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.368327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.369157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:47.369558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.371171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea737b4e-ee59-4e23-a97e-83663cfcba18 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.384965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 379/750] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:47.386261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c952-6a3b-4cb9-ad5a-b7855865beb1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.387849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c952-6a3b-4cb9-ad5a-b7855865beb1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.387849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c952-6a3b-4cb9-ad5a-b7855865beb1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.388114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:47.393314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 380/751] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:47.393314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3c952-6a3b-4cb9-ad5a-b7855865beb1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:47.402583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.403799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.404213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.404676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.405553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:47.406013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.407686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e38a4a-746f-4b82-bca6-af1960820ac3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.420887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 372/752] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:47.422360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.422786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.423223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.442059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:47.443476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:47.456646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:47.521535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3c952-6a3b-4cb9-ad5a-b7855865beb1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.531726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 373/753] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:47.532706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:47.564840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.565929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.566831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.567224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.568485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=103b5c20d7354245a2c57adb0363ffce) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:47.568910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.571274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.581097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 381/754] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:47 2022] GET /identity/v3/role_assignments?user.id=103b5c20d7354245a2c57adb0363ffce&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:47.658245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99d3c2-ad0e-4890-87ba-db47d67a5f23 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:47.675867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:47.703420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:47.704079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:47.704357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:47.704666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:47.705407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=aae0056168e0416c82c5cdba519b9db6) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:47.705706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:47.707140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:47.713786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 374/755] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:47 2022] GET /identity/v3/role_assignments?user.id=aae0056168e0416c82c5cdba519b9db6&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:47.908835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:48.041733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8f5d8-1c31-43c0-b93f-d009970d2e73 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:48.063320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:48.093884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:48.094532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:48.094811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:48.095083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:48.112480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:48.117846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:48.118790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:48.119671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 382/756] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:48.125291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:48.259764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5713f-ee49-4c6c-8588-c2cea4cac166 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:48.300746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:48.301347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:48.301621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:48.301882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:48.302540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:48.302836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:48.304214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:48.320611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 375/757] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:48.325962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:48.477638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca64cf-5758-4832-9953-c6afa727ff79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:48.526145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:48.526961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:48.527238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:48.527508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:48.539069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:48.540547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:48.561842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:48.562837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:48.563783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:48.563783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:48.563783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:48.563783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 383/758] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:48.568528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:48.663009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:48.742709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e505991-29cd-44a8-b918-7703bc172d4c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:48.789993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:48.790524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:48.790803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:48.791073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:48.791749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:48.792053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:48.793413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:48.811338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 376/759] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:48.816406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:48.877757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:48.993163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08ae4-79c6-4d33-960a-dc344094519a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.041010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.041591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.041872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.042421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.051759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.053257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d8951f3d-f9be-4cf7-980d-47bd951adb02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:49.071239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:49.072290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:49.073242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:49.077067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 384/760] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:49.082680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.180616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a847-51d8-45a5-b021-8a5aff261d0b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.180943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a847-51d8-45a5-b021-8a5aff261d0b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.181170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a847-51d8-45a5-b021-8a5aff261d0b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.182500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 385/761] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:49.188203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.188495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.188773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.210661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483e57-e70a-4669-9b03-039e30b81e77 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.224203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:49.244574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:49.245805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:49.249668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.250296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.250563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.250839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.251598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:49.251915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.253348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.272419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 377/762] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:49.279912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.381977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 386/763] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:12:49.444651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159308-05b3-4466-90bf-db47952f8521 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.500849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.501531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.501810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.502079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.523317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.524786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:49.543321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:49.545220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:49.546220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:49.546220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:49.546220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:49.546220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 378/764] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:49.550982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.569567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:49.653125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbe09c-413c-4e26-8768-30294c5546cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.689058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.689665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.689941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.690212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.690851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:49.691151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.692552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.705141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 387/765] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:49.710524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.728615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.730656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:49.823941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:49.846533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d0d8c-ce40-4f8c-8e1e-85937ad62686 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.883100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.883763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.884042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.884311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.884886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.886374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9d47f584-f3c3-4a6e-bab3-61bc401a4128 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:49.897441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:49.898435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:49.899374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 388/766] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:49.908935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb10b1-f061-49fa-8e05-f5280cf3955e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.909215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb10b1-f061-49fa-8e05-f5280cf3955e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.909493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb10b1-f061-49fa-8e05-f5280cf3955e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.911002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 389/767] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:49.913592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb10b1-f061-49fa-8e05-f5280cf3955e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:49.941417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ff9f-5ab9-4564-a610-3c9555248e6f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:49.986989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:49.987669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:49.987938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:49.988209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:49.996939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:49.999612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-531d10b4-3477-48a4-9b45-6e1033704f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:50.013812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:50.014732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:50.015672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:50.019768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 379/768] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:50.021577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.021859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.022130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.039824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb10b1-f061-49fa-8e05-f5280cf3955e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.046242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:50.047443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:50.079815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.081304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.082290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.083257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:50.083790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.085447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.096036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 390/769] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:50.097559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.104782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.143624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 380/770] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:50.144893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.150540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.242740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6e314-461e-4732-9fc5-fde7d1dfc38f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.280794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.281464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.281751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.282029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.282719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:50.283027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.284541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13515e-f2ba-48c3-96f1-78289cfc6366 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.296717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ca148-6903-484d-813b-daa62a566c3f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.304959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 391/771] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:50.304959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697207-df05-42df-8fa7-122d523a60da None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.304959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697207-df05-42df-8fa7-122d523a60da None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.304959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697207-df05-42df-8fa7-122d523a60da None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.307927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 392/772] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:50.308717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697207-df05-42df-8fa7-122d523a60da None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.339294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.340261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.340757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.341866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.342692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:50.343221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.344803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acca-c811-45be-9855-24de33b25cdc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.350449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 381/773] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:50.351887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.352383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.352862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.370851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:50.372496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:50.382933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.438697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.440764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697207-df05-42df-8fa7-122d523a60da None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.455619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 382/774] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:50.457325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.497637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.498417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.498691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.498959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.501103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=dac985d2461d4132af309ded8c7ce1a1, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.502672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.517492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 393/775] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:50 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:50.518405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.646698 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb273c8-f1de-483e-9706-05e1cf38d3b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.677714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.695733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.695733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.695733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.695733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.696145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.696145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-59f7ddfe-e7aa-4644-8b02-d6c2c41415f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:50.708423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:50.709893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:50.710947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:50.718209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 394/776] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:50.719873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa2c07-d2b1-45f8-9fb3-f0ce35c3a86a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.720418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa2c07-d2b1-45f8-9fb3-f0ce35c3a86a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.720796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa2c07-d2b1-45f8-9fb3-f0ce35c3a86a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.722546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 395/777] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:50.723408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa2c07-d2b1-45f8-9fb3-f0ce35c3a86a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.743981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd721a-6873-498e-af2e-42bdc3ef45b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.794584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.795620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.796095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.797116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.798319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] RBAC: Adding query filter params (interface=public, service_id=b4319765cbc64bb385eb5cb77e429877, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:50.798783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.801376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e6d8-e4a3-42e8-bd32-3bf63f21a2be None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.812929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 383/778] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/endpoints?service_id=b4319765cbc64bb385eb5cb77e429877&interface=public®ion_id=RegionOne => generated 194 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:50.814779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.815921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.816199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.838858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:50.840094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:50.849768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa2c07-d2b1-45f8-9fb3-f0ce35c3a86a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:50.887151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:50.888252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:50.888626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:50.888906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:50.889822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:50.890129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:50.891497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:50.897086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 396/779] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:50.897893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.904182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:50.917846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 384/780] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:50.931556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:50.970603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.031046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64da81-cace-415b-9486-38ae6f42ef7f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.048074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ad734-7ed6-41f6-b894-8025b75178ae None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.077466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.078455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.078809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.079563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.087866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.088557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.088908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.089253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.089937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.092260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.093464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:51.097986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 385/781] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.098948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:51.100825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:51.101769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:51.102631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:51.106530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 397/782] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:51.116626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.169190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.221656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793ae39-0029-44df-b0eb-27ca401dd51f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.260639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.261630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.262118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.263220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.264320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:51.264881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.266480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.272899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 386/783] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.280041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.325741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67aefe-8da2-450f-be20-2c22098a7c04 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.343104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.374578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.375199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.375481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.375795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.383557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:51.383557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.383557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.391071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 398/784] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.392018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.412213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72da4-6a91-4631-a8ef-1400810abeb0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.430820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.449103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.449711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.450071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.450355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.451378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] RBAC: Adding query filter params (interface=public, service_id=658ddc2b86e34ef097906b44fa0d3d58, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:51.451727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.453099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.458691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 387/785] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/endpoints?service_id=658ddc2b86e34ef097906b44fa0d3d58&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.459515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.506820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b23282-35e2-422a-a201-52419cb50b81 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.542768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.543399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.543699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.544648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.555448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:51.561708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:51.564020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:51.566100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:51.571721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 399/786] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:51.571721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.572040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632630b9-4a3e-4619-8e0f-1e265cc18a69 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.605840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.608157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.608735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.609013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.609290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.621288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.629346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:51.649379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:51.652142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:51.653433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:51.653433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:51.653433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:51.653433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 388/787] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:51.653433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.688554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4f43a-0054-43c3-a6ac-1cf0f4129d71 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.701607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.733549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.734580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.734958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.735474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.736529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.738485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.747709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 400/788] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.748558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.823175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7125d8-afa9-41db-af74-b03bdac79251 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.891776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.892400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.892680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.892953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.893629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:51.893927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.895291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.903435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:51.908146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff0d0a-0914-4822-b2dd-9a79dd744fbe None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:51.913714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 389/789] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.921782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:51.952126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:51.952881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:51.953161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:51.953430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:51.954083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:51.954379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:51.955803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:51.969732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 401/790] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:51.975515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.083805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c029d24-08cf-4147-8e75-4d5c1cd29669 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.097697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.105016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a9287-151f-4e0d-9a5b-f5a077ec0cee None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.129819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.130431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.130881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.131157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.148289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.150221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.150523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.151232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.151871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.152214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.165638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:52.166804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:52.167799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:52.168852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:52.170100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.172595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 402/791] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:52.179517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:52.185062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:52.186223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:52.187310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:52.187310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:52.187310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:52.189438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 390/792] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:52.196569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.205955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.347217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a79f5-96a9-475b-b731-28f45fb3f524 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.369046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042dca-51cd-4590-b781-3a2181f30417 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.408785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.409676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.410107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.410543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.411336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:52.411827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.413512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.435758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 391/793] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:52.439642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.440428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.440932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.441982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.442864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.443796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:52.444672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.452359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.464001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.492206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 403/794] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:52.498070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.555502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbced4-61f4-41a8-8270-bce149b7d56b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.593523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.594442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.594916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.595335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.603270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.604892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:52.618338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:52.619561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:52.620744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:52.623074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 392/795] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:52.629988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.704170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.741633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.742611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84be-01b0-4121-bfd2-8211635a9bdb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.782544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.783604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.784128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.784552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.785469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:52.785934 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.787593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.809245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 393/796] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:52.816108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:52.828060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2802-9a03-4834-b73f-d2f191aab51f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.929150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7355c-311f-479a-9d84-b49ec0188164 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:52.944309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.946447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.946922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.947860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.966091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:52.967090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.968726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.978038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:52.978989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.979419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.979881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:52.992255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:52.993889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-dc3196c3-7672-4a2a-a94e-11b0401182c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:52.995443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:52.996610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:52.998559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:52.998559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:52.998559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:52.998559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 404/797] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:52.998881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377b5b-281f-4b9c-a886-3e37189cd9dc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:52.999000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377b5b-281f-4b9c-a886-3e37189cd9dc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:52.999243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377b5b-281f-4b9c-a886-3e37189cd9dc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.005494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 405/798] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:53.006903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377b5b-281f-4b9c-a886-3e37189cd9dc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0bc7fb1d-e9a4-402c-936c-246451711f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:53.015518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:53.016684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:53.017692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:53.017692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:53.017692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:53.019399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 394/799] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:53.020883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.021332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.021754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.041984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:53.043491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:53.127645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 395/800] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:53.128803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.149635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377b5b-281f-4b9c-a886-3e37189cd9dc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.191990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.208193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.208788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.209061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.209329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.209969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:53.210263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.211653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.241694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 406/801] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:53.247569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.280757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7690d-3947-4093-96a1-8ea43c9677f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.290886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.335719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.336748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.337304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.337879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.338863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:53.339387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.341241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.369449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 396/802] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:53.376039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.439273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608857c1-b0a9-4e13-b7ea-192d32071345 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.495634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.496629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.497695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.498201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.505920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.507376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:53.519073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:53.520024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:53.520931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 407/803] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:53.521421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.537389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470ca14-1ecc-41c4-b131-90b001005b1b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.570690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.574523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.575391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.575663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.575960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.583572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.584893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-96b71b69-243a-4579-94f6-833046e82598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:53.596078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:53.596998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:53.597907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:53.601822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 397/804] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:53.603073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08944-dca4-4aef-b0f2-e11079b73c55 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.603332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08944-dca4-4aef-b0f2-e11079b73c55 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.603503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08944-dca4-4aef-b0f2-e11079b73c55 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.604765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 398/805] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:53.608148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08944-dca4-4aef-b0f2-e11079b73c55 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.631127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded93a5-0297-476e-9aec-cd118259ba05 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.641463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.669348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.670000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.670387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.670738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.681536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:53.686945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:53.687911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:53.688769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 408/806] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:53.689469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.741404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08944-dca4-4aef-b0f2-e11079b73c55 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.782496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.783334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.783912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.784333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.785257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:53.785712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.787238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a21b75-c193-43e9-86d4-a4be5067d04a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.806104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 399/807] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:53.806279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a35e5c-284d-4e25-8903-4c3dfeaf5fed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:53.806383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.806383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.806383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.812310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.824642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:53.826330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:53.838390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:53.838982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:53.839261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:53.839527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:53.840256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:53.841087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:53.842521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:53.859641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 409/808] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:53.859641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.915560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 400/809] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:53.915560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:53.934970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:53.990649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001e1f-d3a0-4afe-8d53-577ae7ff7e79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.033671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4af70b-e14c-4a72-81ba-79f11edde09e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.044079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.045093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.045469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.045841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.047059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:54.047786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.049748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.059660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 410/810] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:54.060632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.085342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.086293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.086576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.087282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/aae0056168e0416c82c5cdba519b9db6/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.093867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.106181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=aae0056168e0416c82c5cdba519b9db6, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.107711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.122987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 401/811] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:53 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/aae0056168e0416c82c5cdba519b9db6/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:54.123892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.182529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f5ae7-3ac8-4d51-8e3c-f8e519033341 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.208583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.229159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.230179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.230667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.231741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/103b5c20d7354245a2c57adb0363ffce/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.234477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=103b5c20d7354245a2c57adb0363ffce, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.236416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.261565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 411/812] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:12:54 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/103b5c20d7354245a2c57adb0363ffce/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:12:54.267407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ebd76-c3ba-4679-a390-1451ae300b89 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.279262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.329150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.329757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.329998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.330265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.354659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.356220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:54.381728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:54.382824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:54.383793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:54.383793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:54.383793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:54.383793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 402/813] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:54.390849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.417045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.431693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195da4-aa54-461a-b97a-34d48dd428fa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.476313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.476953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.477228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.477495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.478084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.479474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0d554dd8-29b6-4a5c-8c79-d703dcb4651d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:54.492463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:54.493726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 21:12:54.494761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:54.497736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 412/814] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:54.499218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f69c5-fb0f-4171-9d4f-fcba36813073 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.499693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f69c5-fb0f-4171-9d4f-fcba36813073 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.500118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f69c5-fb0f-4171-9d4f-fcba36813073 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.501788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 413/815] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:54.513253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.514098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.514561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.519884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb251f-5d11-4a5e-a198-c84cdd60393e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.537881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:54.539500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:54.551987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.561822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.562565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.563184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.563655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.564829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:54.565244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.566729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.587815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 403/816] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:54.588921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.621124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 414/817] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:54.622860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.696798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aaea5e-5240-49eb-8635-9461613a7f69 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.733737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.734361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.734642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.734917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.735991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:54.735991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.737502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.742930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.743301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 404/818] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:54.748774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.780144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6e096-fd84-4722-827c-888821a7a4a3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.844419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.845372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.847252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.847364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:54.847825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.865727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.868864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:54.887113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:54.888349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:12:54.889404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:54.889945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 415/819] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:54.904964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:54.938552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee72a-cddd-499e-bf01-73bef0f4ed17 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:54.979566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:54.980353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:54.980635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:54.980905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:54.982243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:54.982725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:54.984176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:54.996277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 405/820] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:54.999128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.058438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53b6ed-d6e6-481a-9776-06a1a8dcfd0e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.105473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.107044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.107935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.108205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.108948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:55.109250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.110679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.125687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1b594-662a-4259-acb9-f819bdec7bcf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.134032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 416/821] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:55.134032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.134730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.144518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.175351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.176364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.176851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.177328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] PATH_INFO: `/v3/services/network` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.178195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.179868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:55.189838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:55.190854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 21:12:55.191722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:55.194568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 406/822] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:54 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:55.196131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.253145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e07d-44e5-4394-84c9-62c5752d416e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.295835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.297273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.297972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.298672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.300004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.301676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.324459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb9853-1fb2-4076-b9bc-3f79c8273d25 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.336141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 417/823] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:12:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:12:55.336743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.411700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.412319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.412594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.412863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.468198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.499645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.556020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ed08-87e9-4a85-b415-44cf7e97476c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.558719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.560435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.605389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.606898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.607392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.607942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.608787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:55.609296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.611426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.618904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 418/824] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:55.627122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.712382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 407/825] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:12:55.718985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.757522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c716c-63e7-4654-afc4-ab9698f43f40 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.794648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.823303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.824196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.824568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.825512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.826229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:55.826626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.828142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.846019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 419/826] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:12:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:55.850749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:55.851614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:55.854690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6218f-6eb9-48e6-9cd0-95b4b6d170df None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:55.920785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:55.921426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:55.921854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:55.922250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:55.936864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:55.938381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:55.956133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:55.957848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:55.959029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:55.959029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:55.959029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:55.959029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 408/827] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:55.963638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:56.078391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36754e2f-cea8-4dd2-8647-2b8d37ad85bb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:56.107073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:56.119201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:56.119844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.120135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.120691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.121362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:56.121704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:56.123356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:56.137724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 409/828] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:56.143291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:56.145711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6b69-4bb3-4416-8b74-2f4ea2e5fc21 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:56.252926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34034262-cb94-4ea6-827d-8b256ec0f89f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:56.272264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:56.272892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.273175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.273452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.274136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:56.275558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:56.289684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:56.290731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.291019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.291296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.299424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:56.300916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:56.315165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:56.316530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:56.317860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 410/829] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:56.322954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:56.347394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 420/830] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:12:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:12:56.395969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:56.438548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38145-ff40-490c-8ffc-c1fb75e44524 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:56.483322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:56.483973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.484253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.484528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.485595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:56.485595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:56.486920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4b6f-352a-4822-af1c-b4d871b7aaa2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:56.500604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 411/831] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:56.507238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:56.562445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:56.632258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73482d4a-34dc-42c5-a35b-e988741d4a8a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:56.650134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448848-4664-4295-8a33-e1f2a52743e1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.650454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448848-4664-4295-8a33-e1f2a52743e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.650679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448848-4664-4295-8a33-e1f2a52743e1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.652927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 412/832] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:12:56.660558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dde88-3c1c-45c1-b3e2-165e1ccc31f4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.661055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dde88-3c1c-45c1-b3e2-165e1ccc31f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.661475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dde88-3c1c-45c1-b3e2-165e1ccc31f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.693087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:56.694011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:56.694442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:56.694862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:56.695878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:56.696356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:56.697982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec193b4-f93b-48f9-89e0-e8aaec58077a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:56.702021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2c1dde88-3c1c-45c1-b3e2-165e1ccc31f4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:56.703766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c1dde88-3c1c-45c1-b3e2-165e1ccc31f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:56.707198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 421/833] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:56 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:56.783700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 413/834] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:12:57.011791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb8cc-78bc-4f89-9e7d-131beaf06dc1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.012078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb8cc-78bc-4f89-9e7d-131beaf06dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.012899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb8cc-78bc-4f89-9e7d-131beaf06dc1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.014956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 422/835] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:57.022069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.022803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.022803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.045765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:57.047273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:57.070749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.092241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb8cc-78bc-4f89-9e7d-131beaf06dc1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.120475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 414/836] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:57.202437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb8cc-78bc-4f89-9e7d-131beaf06dc1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.209956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.241319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.241950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.242228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.242502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.253649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:57.258573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:57.259564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:57.260440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 423/837] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:57.269013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.374999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b442e-c84c-4484-9cc0-369f334368b7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.412223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.412836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.413119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.413405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.414063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:57.414366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:57.415771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:57.424630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 415/838] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:57.426525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.430529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.451780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.544370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd66922-c1dd-4c52-9748-d83b03a627aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.552604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51af6-2f38-4675-bb18-400a29fff383 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.586921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.587637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.587916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.588184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.598509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.599986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.600778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.601260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.601368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:57.602948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:57.618825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:57.623181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:57.624167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:57.625165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:57.625165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:57.625165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:12:57.627566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:57.628499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:12:57.629367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:57.630003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 416/839] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:57.632136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 424/840] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:57.639874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.640383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.673351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.729741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.771105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7540b-bf92-4833-a770-4b5f389e1737 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.775321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6e226-4422-4fcb-ba1e-e2d396d57261 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.818826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.819435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.819740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.820020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.820690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:57.820994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:57.822369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:57.831901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.832787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.833267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.833776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.834688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:57.835194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:57.836942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:57.838222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 425/841] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:57.839117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.854547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 417/842] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:57.859583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:57.939361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:57.953767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d42ff-7def-42ee-8001-aed798c80c70 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.968156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973212-f60e-4022-b889-2e21e79f3c15 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:57.989723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:57.990351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:57.990744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:57.991020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:57.996274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.002903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.003583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.003864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.004137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.012960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.014427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-18518f3f-1147-4824-bfbe-814b7ccde0eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:58.028513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:58.030072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:12:58.031050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.034666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 418/843] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:58.035992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413d869-a864-4a7f-8370-fbc6fdb8d4fd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.036279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413d869-a864-4a7f-8370-fbc6fdb8d4fd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.036551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413d869-a864-4a7f-8370-fbc6fdb8d4fd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.039598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 419/844] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:58.039598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413d869-a864-4a7f-8370-fbc6fdb8d4fd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.116972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.118410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-22519438-cb71-4386-a6a5-c2e9b5b00550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:58.135174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:58.136371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 426/845] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.137883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.138267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.156906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:58.158098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:58.159463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413d869-a864-4a7f-8370-fbc6fdb8d4fd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.196729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.197258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.197536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.197806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.198601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:58.198913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.200690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.224383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 420/846] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:58.224383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.243791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 427/847] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:58.244544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.275602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.321069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.334103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cac8f-6334-416f-9660-2706fa95a104 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.355188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2d9e4-31bf-4b82-9eda-bf7410b73081 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.372098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.373011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.373543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.374164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.375125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:58.375666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.377282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.393882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.393882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.393882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.393882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.394698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 421/848] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:58.395869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.468574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.469969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.506010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a44e20-c2cf-47d9-b0bd-c0607319cc4f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.516820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.543608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.545421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.546065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.546342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.546673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.555919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.557342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:12:58.568632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:58.569775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.570725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 422/849] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:58.571328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.583131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 428/850] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:12:58.584716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.688617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac78a-ac29-4c71-a03e-40e78b15d116 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.700423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84bfc2-4305-497a-ad8d-d61d4a1383c8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.725861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.726409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.726692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.727220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.727563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.728987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.738068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:58.738306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:58.739264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:12:58.744719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.745479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.745479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.745479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.745675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 423/851] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:58.745744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:58.745744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.745867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.745934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.761409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 429/852] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:58.763037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.784826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.794080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:58.863237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e517b1-e32d-4abe-923b-70c6c02d992f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.877888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1f93-8942-4800-ba5e-e3e95c2f7d4a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:58.901051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.902216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.902500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.903325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.916194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:58.916819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:58.917164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:58.917266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.917438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:58.918051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:58.918448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:58.919246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.920575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:58.925257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 430/853] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:58.926338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:58.935379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:58.936394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:58.937390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:58.937390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:58.937390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:58.937390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 424/854] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:58.937390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.037235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.037840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.052834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38490d69-c9ee-4bba-8362-c743572607b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.061300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5691-6fc7-4e6d-96ec-8193c8521b84 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.094659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.095265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.095571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.095842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.100293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.100895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.101171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.101440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.102062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:59.102359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.103745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.108115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 425/855] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:59.108885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.172763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.175383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0bc27-baf5-4059-88e8-c39a425c0777 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.222591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c82d2f-40c9-4127-b67e-22b86d7a5769 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.262771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.263789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.264247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.264715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.265542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:59.266051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.267501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ea304-a2b7-49e1-a929-f3855f88145a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.281257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.283494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 426/856] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:59.283655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7bfd-9338-4dbb-bafd-c3af64fedebb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.283898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7bfd-9338-4dbb-bafd-c3af64fedebb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.284307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7bfd-9338-4dbb-bafd-c3af64fedebb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.285612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 427/857] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:12:59.287716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f7bfd-9338-4dbb-bafd-c3af64fedebb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.298957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 431/858] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:12:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:12:59.300183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.300462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.300730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.301434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.320468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:12:59.321640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:12:59.404521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 432/859] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:12:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:12:59.405248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.406082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f7bfd-9338-4dbb-bafd-c3af64fedebb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.442178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.442964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.443310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.443665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.450770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.452329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:12:59.464037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:59.464987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:12:59.466354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 428/860] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:12:59.470696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.520216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b761a-3d9f-40a6-9b7a-2eb97619d708 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.542685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.550324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.568116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.569087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.569665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.570123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.581594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b2334-ecd7-4c19-89be-597b734dbc73 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.582022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.583923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:12:59.600522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:59.601619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:12:59.602548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:12:59.602548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:12:59.602548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:59.604797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 433/861] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:12:59.610415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.623835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.627350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.640300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 429/862] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:59.645710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.736068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22c829-a4c7-4f13-bb01-5c417d263e87 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.760135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041a0c-013a-42ae-8e22-1a5be191b141 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.777357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.778829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.779305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.779619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.780265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:59.780565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.781924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.788737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.799249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 434/863] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:12:59.799249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.803103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.803747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.804027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.804301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.805034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=aae0056168e0416c82c5cdba519b9db6) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:12:59.805342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.806763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.813831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 430/864] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/role_assignments?user.id=aae0056168e0416c82c5cdba519b9db6&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:12:59.814421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:12:59.841602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:12:59.920222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280caa-c5df-450f-b9c9-cefcb7ec815b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.960484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a271b9-7fc0-44cc-8068-aecd19f5b79c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:12:59.969336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:12:59.971014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:12:59.971508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:12:59.972017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:12:59.973297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:12:59.975815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:12:59.989906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:12:59.991034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:12:59.992065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:12:59.998471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 435/865] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:00.002142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:00.012709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:00.014478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:00.014758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:00.015033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:00.022632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:00.024124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:00.037666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:00.039045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:00.040089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 431/866] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:12:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:00.044897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:00.119867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:00.145216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:00.149974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150fe1b-fbc9-46ed-ab73-887f2728efd0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:00.174677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3fecf-2158-4754-9b83-83c65ce48451 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:00.187517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:00.188372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:00.188907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:00.189338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:00.190188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:00.190647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:00.192289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:00.197639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 436/867] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:00.203575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:00.210312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:00.211007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:00.211292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:00.211590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:00.212355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:00.212664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:00.214085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:00.226973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 432/868] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:00.233722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:00.319981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1d01-52dc-4ae4-8f3b-fd8b779fcf8f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:00.352209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca67a56-6e54-4a1b-bbe1-7c5e95b78d33 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:00.360768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:00.361934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:00.362226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:00.362502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:00.363198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:00.363509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:00.365836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a2ddb-cda5-41e0-8790-d1a3916c4257 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:00.368496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:00.374302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 437/869] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:00.401793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:00.403169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:00.403902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:00.404500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:00.405387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=103b5c20d7354245a2c57adb0363ffce) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:00.405799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:00.407484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1e328-c141-4ace-a6ca-bbb6885408fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:00.415096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 433/870] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:00 2022] GET /identity/v3/role_assignments?user.id=103b5c20d7354245a2c57adb0363ffce&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:01.311262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae871be-2914-4989-930c-f323bbb84aad None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:01.311865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae871be-2914-4989-930c-f323bbb84aad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:01.311865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae871be-2914-4989-930c-f323bbb84aad None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:01.313554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 438/871] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:01.319611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60cd5-c05a-4b3a-adbf-9f9d59ced765 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:01.320119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60cd5-c05a-4b3a-adbf-9f9d59ced765 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:01.320218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a60cd5-c05a-4b3a-adbf-9f9d59ced765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:01.338662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-89a60cd5-c05a-4b3a-adbf-9f9d59ced765 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:01.339876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-89a60cd5-c05a-4b3a-adbf-9f9d59ced765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:01.417477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 434/872] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:01.683418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae871be-2914-4989-930c-f323bbb84aad None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:01.713284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:01.794747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae871be-2914-4989-930c-f323bbb84aad None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:01.830097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:01.830791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:01.831072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:01.831346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:01.832103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:01.833477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a157-97cd-4eac-b5d1-4d4ecb601be6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:01.850702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 439/873] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:01 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:02.227975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf436e6-886b-472e-9d69-f46e515ed440 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.228696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf436e6-886b-472e-9d69-f46e515ed440 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.229202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf436e6-886b-472e-9d69-f46e515ed440 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.231659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 435/874] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:02.237297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d7b6-c9d3-4a36-bbce-401e32206a4b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.237861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d7b6-c9d3-4a36-bbce-401e32206a4b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.238362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006d7b6-c9d3-4a36-bbce-401e32206a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.263341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f006d7b6-c9d3-4a36-bbce-401e32206a4b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:02.264613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f006d7b6-c9d3-4a36-bbce-401e32206a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:02.342535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 440/875] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:02.620875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf436e6-886b-472e-9d69-f46e515ed440 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:02.629129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935dc7-a807-43a5-b9a0-1b809a1cf0c8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.629498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935dc7-a807-43a5-b9a0-1b809a1cf0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.629676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935dc7-a807-43a5-b9a0-1b809a1cf0c8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.631815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 441/876] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:02.641000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826c353-a8a7-4d62-8be1-735c7b9aff50 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.641343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826c353-a8a7-4d62-8be1-735c7b9aff50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.641555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826c353-a8a7-4d62-8be1-735c7b9aff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.662740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b826c353-a8a7-4d62-8be1-735c7b9aff50 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:02.664123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b826c353-a8a7-4d62-8be1-735c7b9aff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:02.673616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:02.752061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 442/877] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:02.754714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877825-f1e3-410b-a66c-f3b375fa5f5f None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.755004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877825-f1e3-410b-a66c-f3b375fa5f5f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.755281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877825-f1e3-410b-a66c-f3b375fa5f5f None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.760363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 443/878] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:02.760499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf436e6-886b-472e-9d69-f46e515ed440 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:02.761664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.762138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.762573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.774270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:02.783677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:02.784907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:02.809997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:02.811264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.812230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.812861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.825143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:02.830176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:02.831111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:02.831996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:02.834740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 436/879] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:02.841650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:02.875771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 444/880] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:02.959312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4308df-6cc3-44c0-8267-6013a12b2de4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:02.994770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:02.995642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:02.996064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:02.996541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:02.997340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:02.997783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:02.999270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.008015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 437/881] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:03.014322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.024860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.061743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.115711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.140149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695fc9d-c186-4a03-aa36-6c3176575fc0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.147627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebfe16-eec3-4c19-809c-173a39319039 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.175799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.176507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.176791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.177068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.177720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.179079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.185579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.186342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.186646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.186915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.195032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 438/882] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:03 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:03.198537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.201080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.204605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:03.219367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:03.220449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:03.221356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:03.221356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:03.221356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:03.225542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 445/883] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:03.226538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.334205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.357488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5c101-833d-451c-9c62-16a60d207cb9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.360984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e0194-8bb8-4d7d-b76d-d61aa0178345 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.393874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.403525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.404177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.404454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.404720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.405300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.406649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.407109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.407920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.408384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.408533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.409180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:03.410722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.410838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:03.417509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:03.419049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:03.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 439/884] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:03.425401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.427575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 446/885] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:03.433486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.542146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa1a05-c6f5-49f9-ac29-26d1777908c5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.569052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019ff6-ea89-4a64-8451-e4f8856360e3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.579111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.580035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.580514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.580967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.581778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:03.582274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.583888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.589958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 440/886] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:03.595916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.611308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.619700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.620343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.620623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.620897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.641878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.643493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:03.661076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:03.662075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:03.663059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:03.667162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 447/887] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:03.669389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.719782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c98-ddf7-4533-aa74-ab17c5227cda None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.725850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.759554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.760483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.760946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.761423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.762882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:03.763366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.765396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.771716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 441/888] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:03.801604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1d5f6-d96f-4841-ac41-47a3696c032c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:03.841064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:03.842163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:03.842624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:03.843080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:03.844076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:03.844592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:03.846353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a1c03-8858-4826-ad59-989bf8458959 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:03.867789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 448/889] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:03.876894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:03.974198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:03.999016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a87e43-ab12-4217-bddf-b8d97e174d81 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.041038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.041651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.041928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.042199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.114726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.116257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.129147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430961-6454-4dc3-a4e7-9b6783a0fab5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.129483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430961-6454-4dc3-a4e7-9b6783a0fab5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.129697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430961-6454-4dc3-a4e7-9b6783a0fab5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.131207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 449/890] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:04.137067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.137571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.137925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.166563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:04.169511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:04.210111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:04.241841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 442/891] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:03 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:04.247987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.263580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 450/892] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:04.361993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227e447-58aa-47bf-8c06-182b49961b02 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.397938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.398889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.399333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.399806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.410864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.412562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:04.427820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:04.428980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:04.430119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:04.430119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:04.430119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:04.431921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 443/893] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:04.440702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.491978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:04.540945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.546728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638637f-d880-459d-807a-f97cb064a1eb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.572838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:04.595861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.596751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.597315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.597746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.598840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:04.599305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.600935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.617954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 451/894] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:04.624372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.660992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c538a5-28f9-4730-8041-106250a4c859 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.697181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.697825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.698294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.698413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.698933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.700457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:04.709590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:04.710588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:04.711428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:04.715808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 444/895] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:04.716685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.745201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:04.758385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2af25-31ce-4223-bbeb-826ae65932ff None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.797266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.797905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.798186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.798463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.805916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.807424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:04.820084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:04.820989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:04.822164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 452/896] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:04.827162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.834633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:04.846573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc828d-e8d1-4c08-a676-88af2979a3aa None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.891884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.893161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.893937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.894410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.895406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:04.896271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.899244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.911524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 445/897] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:04.919792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:04.944390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b21ae-1ee0-4617-83b9-f7ff9b54206b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:04.980216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:04.981584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:04.981977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:04.982134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:04.982816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:04.983123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:04.984642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:04.997617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 453/898] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:05.004645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:05.028302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:05.045796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b80c9-19cb-4907-97d5-d0fe1ea6f7cb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:05.089258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:05.091149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.091603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.092031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.092805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:05.093588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:05.095719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:05.102488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 446/899] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:05.110487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:05.128038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e553558-6e6d-4cdc-8477-31af7761dfef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:05.139559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:05.173208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:05.174344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.174828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.175301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.176304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] RBAC: Adding query filter params (role.id=338b17a9779a44a38b33c0ca5107a2f3, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=aae0056168e0416c82c5cdba519b9db6) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:05.176834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:05.178549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:05.191887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 454/900] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:05 2022] GET /identity/v3/role_assignments?user.id=aae0056168e0416c82c5cdba519b9db6&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=338b17a9779a44a38b33c0ca5107a2f3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:05.229953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129081-6e08-48c8-b8f5-494dab49e8d9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:05.268880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:05.269403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.269678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.269944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.270657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] RBAC: Adding query filter params (interface=public, service_id=c9f1274059684185840155e1cf1bcd08, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:05.270964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:05.272370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126ed6c-5482-45e2-a0bc-68c212731aae None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:05.279499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 447/901] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:13:05 2022] GET /identity/v3/endpoints?service_id=c9f1274059684185840155e1cf1bcd08&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:05.284805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:05.372622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:05.398182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb12a54-52a8-4ddb-9e0c-be45475a0f0c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:05.440778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:05.441449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.441851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.442132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.442754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:05.444180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e22fc-2219-4a40-a3df-9b0d38aad9a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:05.451364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 455/902] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:05 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:13:05.539052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2ddc3-ca1e-46d3-91d1-aa1089508f68 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.539357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2ddc3-ca1e-46d3-91d1-aa1089508f68 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.539526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2ddc3-ca1e-46d3-91d1-aa1089508f68 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.541225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 448/903] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:05.547135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:05.547438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:05.547740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:05.567814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:05.569023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:05.648222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 456/904] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:05.916618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2ddc3-ca1e-46d3-91d1-aa1089508f68 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.011641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:06.029853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2ddc3-ca1e-46d3-91d1-aa1089508f68 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.066778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.067420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.067726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.067998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.068582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.069938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-495b2e86-a45e-4c82-a5f9-9eccc19539ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:06.079008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:06.079974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:06.080886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:06.083761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 449/905] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 21:13:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:06.086613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.088901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6f31-206a-4a3e-bb5b-e14daa0a124d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.089412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6f31-206a-4a3e-bb5b-e14daa0a124d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.090061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6f31-206a-4a3e-bb5b-e14daa0a124d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.091741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 450/906] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:06.100015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.100312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.100577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.118844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:06.120125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:06.175106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:06.208361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289441c-8bfb-4644-900d-cf2db4006d94 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.215166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 451/907] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:06.254232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.254834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.255111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.255674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.256306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:06.256603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.257952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.263075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 457/908] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:06.268886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.350204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:06.373728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a44e76-e2f5-4b20-baf1-ce21f6aaae9d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.411994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.412911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.413360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.413774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.414565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:06.415009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.416545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.426721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 452/909] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:06.429596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.499882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.543615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a64e5-335b-4905-b79c-c259af304c89 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.585617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.586604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.587091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.587587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.588480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] RBAC: Adding query filter params (interface=public, service_id=ff97468b54fd4dad9de7c249a17f3fc7, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:06.589006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.591105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.593312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:06.605112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 458/910] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:13:06 2022] GET /identity/v3/endpoints?service_id=ff97468b54fd4dad9de7c249a17f3fc7&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:06.607223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:06.620492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d95f4f-0c42-4677-9bc5-ad09d494e90b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.660106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.660711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.661043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.661317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.661963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.663307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf39760-d189-4cfc-b680-2b95a12e28ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.687666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 453/911] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:06 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:06.740441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8273cd0-7cb6-47e1-bf30-bae94fc593ef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:06.793386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:06.794265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:06.794754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:06.795312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:06.796151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:06.797566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663739-f4b8-4975-b27e-ee404b36af62 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:06.804782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 459/912] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:06 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:13:07.473156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8afc5-8b9c-415f-8aad-41df45adc4d7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:07.474139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8afc5-8b9c-415f-8aad-41df45adc4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:07.474623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8afc5-8b9c-415f-8aad-41df45adc4d7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:07.476443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 454/913] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:07.482334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f944ec-4736-48b8-961d-58895f76851e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:07.482954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f944ec-4736-48b8-961d-58895f76851e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:07.483430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f944ec-4736-48b8-961d-58895f76851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:07.503027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c2f944ec-4736-48b8-961d-58895f76851e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:07.504564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c2f944ec-4736-48b8-961d-58895f76851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:07.561241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:07.580016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 460/914] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:07.771467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e613-79f6-45ef-84ea-be969135a93d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:07.772140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e613-79f6-45ef-84ea-be969135a93d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:07.772627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e613-79f6-45ef-84ea-be969135a93d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:07.774441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 455/915] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:07.780504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:07.780788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:07.781064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:07.799378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:07.800580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:07.872810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 461/916] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:07.878222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3e613-79f6-45ef-84ea-be969135a93d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:07.881268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:07.988267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3e613-79f6-45ef-84ea-be969135a93d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.023256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.023834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.024108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.024430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.036331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:08.041191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:08.042129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:08.043185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 456/917] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:08.050499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.150574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.159361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e04a0c-c4f4-4530-b0a9-5684e75d0488 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.178107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.198997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.200168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.200445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.200837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.201528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:08.201895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.203257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.212276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 462/918] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:08.217023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.218628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.256218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2397d-81b0-41c5-a562-1cd6d07719d2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.291107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.291815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.292157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.292444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.293754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.295301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:08.304329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:08.305346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:08.306852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 457/919] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:08.311461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.329228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985649-55ca-479c-b2ef-5190fc2ab8d2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.367157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.367984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.368356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.368721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.380274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.381833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:08.397033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:08.398712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:08.401176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:08.401176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:08.401176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:08.405135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 463/920] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:08.406061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.439472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.441927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51098959-5429-4507-a9c4-dcb7773bbb47 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.476942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.490485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.491885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.492311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.492856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.494082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:08.494530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.497643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.504705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 458/921] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:08.511583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.522092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974e10c-ce41-4380-be37-18fadeeccfe8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.567775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.568389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.568630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.568842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.569535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:08.570318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.572252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.591571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 464/922] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:08.595162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.638372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f402-82a3-4eba-bacf-05447b849c38 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.676392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.677133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.677503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.677870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.678683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:08.679096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.680634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.690703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 459/923] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:08.691753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.721335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.732878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76fb14-0be3-4c66-ac0d-b261e8d15e5c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.752616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:08.769196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.770084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.770583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.771073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.779841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.781578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:08.794528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:08.795596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:08.796589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:08.799116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 465/924] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:08.804590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:08.809136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac2393-e466-4954-9cb3-28e79bb7b23f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.856926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:08.857568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.857856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.858132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.858819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:08.861044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084f396-b18f-4730-9a3d-88d411d8dcce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:08.895634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 460/925] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:08.949474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94dca7-89b6-405e-b4ba-84dca5d17c13 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:08.986731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f7cc-aa94-48df-a066-78b1ba8aebac None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:08.987474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f7cc-aa94-48df-a066-78b1ba8aebac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:08.987992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2f7cc-aa94-48df-a066-78b1ba8aebac None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:08.990118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 461/926] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:09.001271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc926f-8e61-47db-a033-4f37305e2918 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.001616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc926f-8e61-47db-a033-4f37305e2918 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.001842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc926f-8e61-47db-a033-4f37305e2918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.012771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.013820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.014371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.014856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.015948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:09.016463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.019314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.024143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-03cc926f-8e61-47db-a033-4f37305e2918 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:09.025375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03cc926f-8e61-47db-a033-4f37305e2918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:09.045869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 466/927] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:13:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:09.052048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.063450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.116426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 462/928] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:09.117479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2222f8b-492a-4210-b045-b9eab4e09516 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.117768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2222f8b-492a-4210-b045-b9eab4e09516 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.118039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2222f8b-492a-4210-b045-b9eab4e09516 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.119489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 463/929] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:09.125345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.125635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.125913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.147740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:09.148945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:09.173934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c2b18-bdc5-4b90-beae-23b92c8e5720 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.201086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.219203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.219878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.220488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.220768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.231836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 464/930] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:09.234147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.235630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:09.251092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:09.252287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:09.253445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:09.253445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:09.253445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:09.253445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 467/931] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:09.257997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.361398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984852-e8cc-47f6-91d9-dcdbf7a83ed1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.396408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.397236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.397655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.398066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.398858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:09.399312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.401090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.413657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 465/932] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:09.420028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.421028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.457317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.509480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.538770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d56e3-8e01-4303-9365-9d1de9bc59c9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.565632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf39630-20a1-4729-8f12-a5ef987383cb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.578655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.579299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.579623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.580198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.580798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.582340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:09.591620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:09.592637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.593557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 466/933] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:09.594344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.610547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.611174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.611454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.611754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.618652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.620121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:09.638685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:09.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 468/934] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:09.645840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.698574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.707675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dd85d-e67b-40b8-8940-4e9abbbcb8c1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.743192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.744028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.744400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.744866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.745444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.747498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.752253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050d461-2d7d-410f-a40e-b13ab6c06da4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:09.756722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:09.757837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 21:13:09.758946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:09.763566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 467/935] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:09.763566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.775668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.796641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.797435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.797714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.798120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.798731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:09.799037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.800530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.810636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 469/936] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:09.811842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.873470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfffecf-521b-4a07-a561-0e15108f0eac None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.908714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.909330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.909611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.909886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.910594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:09.910906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.912409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.918941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23f74e-6b87-40a8-9c4e-d5e02c60e937 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:09.926485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 468/937] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:09.926485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:09.950856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:09.960982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:09.961588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:09.961867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:09.962144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:09.962765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:09.963069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:09.964531 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:09.969582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 470/938] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:09.970266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:10.021083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:10.046560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2327768-390f-4357-aedc-5e84a83e6211 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.079580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fa26e-1c93-4dc4-b31e-b9f1c2683709 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.085179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.085853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.086197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.086534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.087229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:10.087625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.089038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.094500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 469/939] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:10.095361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:10.120326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.121700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.121986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.122256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/aae0056168e0416c82c5cdba519b9db6/roles/338b17a9779a44a38b33c0ca5107a2f3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.124475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=aae0056168e0416c82c5cdba519b9db6, role_id=338b17a9779a44a38b33c0ca5107a2f3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.125908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.139965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 471/940] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:13:09 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/aae0056168e0416c82c5cdba519b9db6/roles/338b17a9779a44a38b33c0ca5107a2f3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:10.140889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:10.215452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:10.238231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452d280-c824-4631-89f7-f2a8ed025faf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.285071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.286026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.286305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.286574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.287502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:10.287851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.290119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.296319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a50560-a046-4710-a8b3-afe45b669cdb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.297722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 470/941] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:10.303398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:10.305544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:10.345773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.346447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.346729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.347002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.347743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] RBAC: Adding query filter params (interface=public, service_id=4b14f725fe5e4b22bb799c853db67a32, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:10.348054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.349426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.358689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 472/942] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:13:10 2022] GET /identity/v3/endpoints?service_id=4b14f725fe5e4b22bb799c853db67a32&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:10.362842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:10.451011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff0b5-0aa4-49a8-8983-e601badccf22 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.471896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282192-432e-4565-9cd1-220e26711b18 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:10.498213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.499088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.499366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.499891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.500982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.502827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd99106-c414-4825-8463-0b1271c10706 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.509342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:10.510008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:10.510288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:10.510560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:10.511213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:10.513740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc31f65-c98a-4150-8de5-7b0fe6d77c93 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:10.522261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 473/943] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:10 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:13:10.531056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 471/944] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:11.276550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839212-a333-4c18-974c-78d838faad3e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:11.277889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839212-a333-4c18-974c-78d838faad3e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:11.277889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839212-a333-4c18-974c-78d838faad3e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:11.279311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 474/945] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:11.284695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7452-c768-4a43-8557-bed690df051d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:11.285196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7452-c768-4a43-8557-bed690df051d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:11.285369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc7452-c768-4a43-8557-bed690df051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:11.291484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:11.306734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-14fc7452-c768-4a43-8557-bed690df051d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:11.308990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-14fc7452-c768-4a43-8557-bed690df051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:11.383577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 472/946] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:11.657120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839212-a333-4c18-974c-78d838faad3e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:11.695482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:11.766638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839212-a333-4c18-974c-78d838faad3e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:11.803271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:11.804275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:11.804793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:11.805276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:11.806127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:11.807778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:11.811151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING keystone.server.flask.common [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 21:13:11.832093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 21:13:11.832093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 21:13:11.832093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [parameters: {'id': 'e020e968ba604c709dfda6c8a9224f43', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 21:13:11.832093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e020e968ba604c709dfda6c8a9224f43.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e020e968ba604c709dfda6c8a9224f43. Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:13:11.850228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 21:13:11.851118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Sep 09 21:13:11.852142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 21:13:11.853046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 21:13:11.853963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:13:11.854775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'id': 'e020e968ba604c709dfda6c8a9224f43', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:11.858255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:11.860137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e020e968ba604c709dfda6c8a9224f43. Sep 09 21:13:11.862271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:11.864965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 475/947] 10.222.0.46 () {60 vars in 1140 bytes} [Fri Sep 9 21:13:11 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:11.872812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc7452-c768-4a43-8557-bed690df051d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:11.979360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc7452-c768-4a43-8557-bed690df051d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:12.006631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:12.016125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:12.016802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.017090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.017364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.028593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:12.030049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8c5edc9e-a848-42e0-8692-c87cbafa1ca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:12.041403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:12.042284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:12.043282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 473/948] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:12.050185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:12.157033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:12.167056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a999e6-a2b7-4154-8f55-7c816f68f5f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:12.204247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:12.205247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.205722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.206218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.207098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:12.207625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:12.209630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abca533-e7bc-41d0-a80e-0e9eed2dec09 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:12.232276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 476/949] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:12.505504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a6ac-43c4-4fc2-a654-abd321da4531 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.505992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a6ac-43c4-4fc2-a654-abd321da4531 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.505992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a6ac-43c4-4fc2-a654-abd321da4531 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.507587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 474/950] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:12.513482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb63d-ad66-4a79-a5c7-86d25f76d792 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.513913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb63d-ad66-4a79-a5c7-86d25f76d792 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.514012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb63d-ad66-4a79-a5c7-86d25f76d792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.533820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-14fbb63d-ad66-4a79-a5c7-86d25f76d792 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:12.535331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14fbb63d-ad66-4a79-a5c7-86d25f76d792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:12.608286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 477/951] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:12.779965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d8c6-358f-4b50-925a-6e7c4a82da55 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.780156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d8c6-358f-4b50-925a-6e7c4a82da55 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.780422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d8c6-358f-4b50-925a-6e7c4a82da55 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.782006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 475/952] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:12.787045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.787343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.787656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.809644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:12.810851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:12.854666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293067-8ade-4f73-ba7b-f89a3ff95fde None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.855004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293067-8ade-4f73-ba7b-f89a3ff95fde None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.855265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293067-8ade-4f73-ba7b-f89a3ff95fde None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.856913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 476/953] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:12.862104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:12.862394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:12.862674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:12.880863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:12.882062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:12.887581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 478/954] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:12.919318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:12.932271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:12.963293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 477/955] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:13.026440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26160-5d46-4efe-8afd-74cb5d95848c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.063944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.064488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.064759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.065026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.075988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:13.080822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:13.081806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:13.082949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 479/956] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:13.091918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.162297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.210074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bdca1-4e2b-408b-a487-7fc40321f41c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.232447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.251823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.252211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.252431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.252686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.253631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.253631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.253890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.255282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.265836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 478/957] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:13.266585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.296045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6605af-7676-4762-94c0-35fe5726a045 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.336078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.336577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.336860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.337135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.337719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.339093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:13.348272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:13.349349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:13.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 480/958] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:13.351002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.380740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e394-5d6c-422c-83fb-8c5e2469766a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.418962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.419620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.419922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.420166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.420745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.422143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:13.430892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:13.431925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:13.432791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 479/959] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:13.434807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.467857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c855077-58e8-499a-be3b-5df34c9dd35b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.488336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.503482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.504141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.504419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.504691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.507582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.518073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.519720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:13.534727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:13.536017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:13.536939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:13.536939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:13.536939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:13.536939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 481/960] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:13.537181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.559773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da181e-8a5a-4ffd-8ea0-b348640f7958 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.595976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.596495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.596776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.597042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.597770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.598175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.599520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.607723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 480/961] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:13.607723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.645662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60fbb-61c6-4f20-b274-7e4b371a1296 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.684606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.685220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.685495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.685763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.686801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.686801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.688048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.693025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 482/962] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:13.693851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.721820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110b684-a5d1-47ba-99d9-7fbc1969a3b8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.733620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.746653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.763379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.764013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.764438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.764717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.765839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.766146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.767509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.784227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 481/963] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:13.784227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.805816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5179c0-f459-4f48-b954-b54b3499d934 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.841516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.842413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.842840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.843119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.843833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.844075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.845436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.850438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 483/964] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:13.856800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.887273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3828b09-2989-49d6-bf1d-f834a2489db4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.922470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:13.923066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:13.923341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:13.923645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:13.924340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:13.924638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:13.925978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:13.939583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 482/965] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:13.939583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:13.970722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50e3b-227e-4ea8-b284-28bf2905acf2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:13.975104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:13.987922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:14.010997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.011891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.012274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.012544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.022618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.025353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:14.043876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:14.044922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 21:13:14.045906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:14.049987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 484/966] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:13:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:14.050926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:14.062335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d9ad8-7b52-453e-81bb-885f74ca3ed8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:14.101189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.101948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.102288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.102628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.103497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.105667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:14.131746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 483/967] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:14.184598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf92a-c564-4573-ac87-5f368f5d9212 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:14.232924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.233537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.233816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.234083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.235112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:14.235463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.237007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18b417-4b3b-4d15-bc8e-8eae6567150b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:14.257378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 485/968] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 21:13:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:14.264112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:14.366521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:14.381062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216990b-85f2-465d-81af-9e114a0a1d0a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:14.422005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.422989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.423473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.424024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.501814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.503833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:14.601025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:14.608141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c081e6-5575-4415-ba30-ebc7ec671fa3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.608629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c081e6-5575-4415-ba30-ebc7ec671fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.608725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c081e6-5575-4415-ba30-ebc7ec671fa3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.610026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 486/969] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:14.615204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.615492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.615812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.618350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 484/970] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:14 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:14.624766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:14.640957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:14.642385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:14.674369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:14.727146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 487/971] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:14.755204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8210a42-644b-41d3-be96-87d38b60d94d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:14.797245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.798138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.798598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.799074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.810108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.811823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:14.827207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:14.828473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:14.829570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:14.829570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:14.829570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:14.831317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 485/972] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:14.836924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:14.945800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820ba32-5e5f-4912-8e73-041d3ac1db42 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:14.988202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:14.989228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:14.989664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:14.990093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:14.990925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:14.991401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:14.993049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.006929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 488/973] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:15.011785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.013729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.039523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:15.063949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:15.122313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9a497-756a-4e35-a644-942f3a64e6d0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.153751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68935c20-a5c2-4091-a217-fe2e0d7a74ef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.158055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.158781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.159163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.159548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.160302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.161817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.164481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING keystone.server.flask.common [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 21:13:15.177869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 21:13:15.177869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 21:13:15.177869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [parameters: {'user_id': '69b9aafed5ca4e9c9e55cf061bd91eab', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 21:13:15.177869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 09 21:13:15.188751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:13:15.189676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 21:13:15.190753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 21:13:15.191689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 21:13:15.194151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 21:13:15.196300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'user_id': '69b9aafed5ca4e9c9e55cf061bd91eab', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:15.198415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 21:13:15.201915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.207894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 486/974] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 21:13:15 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:15.208909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.208909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.208909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.208909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.209527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.215036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.216822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:15.230571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:15.231480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:15.232534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:15.234911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 489/975] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:15.240332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.324414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ce24c-bbb7-4167-ae50-809b1c41e69d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.326138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:15.329179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:15.357130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f033d6-5197-476b-8e11-c6c1ae5f626b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.369795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.371527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.372152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.372610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.381041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.382959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.395812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.396508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.396859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:15.397154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:15.398084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 21:13:15.399070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:15.399674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.399674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:15.399674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.399674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.407852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 487/976] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:13:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:15.409404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.415115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 490/977] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:15.421664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:15.529242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0f9e1-f59b-4df8-be40-c77b975457b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.548088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e908f3-374e-47d7-a04f-e6a716ed33f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:15.570615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.571627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.572160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.572676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.574005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] RBAC: Adding query filter params (role.id=338b17a9779a44a38b33c0ca5107a2f3, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=aae0056168e0416c82c5cdba519b9db6) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:15.574615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.576297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab58234-c609-4a9e-9266-eaa4c07bb3f1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.587621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 491/978] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:15 2022] GET /identity/v3/role_assignments?user.id=aae0056168e0416c82c5cdba519b9db6&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=338b17a9779a44a38b33c0ca5107a2f3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:15.595506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:15.596298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:15.596678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:15.597081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:15.597976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:15.598376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:15.600030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce48b3-b59a-46c1-ba1d-106691f9cc5c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:15.654468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 488/979] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 21:13:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:16.327896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bd80-5bb7-4eb3-aef4-53fdc8e8c93e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.328436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bd80-5bb7-4eb3-aef4-53fdc8e8c93e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.328436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bd80-5bb7-4eb3-aef4-53fdc8e8c93e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.330073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 492/980] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:16.335896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d2c8-7530-429f-b269-bf046f371b08 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.336221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d2c8-7530-429f-b269-bf046f371b08 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.336462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d2c8-7530-429f-b269-bf046f371b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.357070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d202d2c8-7530-429f-b269-bf046f371b08 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:16.358622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d202d2c8-7530-429f-b269-bf046f371b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:16.374476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:16.430339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 489/981] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:16.445285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ed12-1d2e-4b98-a3bb-6b26e11c393b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.445588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ed12-1d2e-4b98-a3bb-6b26e11c393b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.445836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ed12-1d2e-4b98-a3bb-6b26e11c393b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.447319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 493/982] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:16.452693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.452979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.453248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.472494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:16.473660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:16.544419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 490/983] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:16.705977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297ed12-1d2e-4b98-a3bb-6b26e11c393b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:16.709137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:16.818726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:16.819273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297ed12-1d2e-4b98-a3bb-6b26e11c393b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:16.854856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:16.855600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.855887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.856163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.856832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:16.858225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:16.874925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 494/984] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:16 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:16.909539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:16.933523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e942598-8f17-44e1-acae-0280ab34a8a9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:16.980902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:16.981902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:16.982191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:16.982524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:16.983638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:16.985873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:16.997787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:16.998920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:17.000378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:17.004196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 491/985] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:17.005563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:17.076032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:17.122409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a226-fb2b-4b1c-9fdf-5e95ee93f3e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:17.157903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:17.158514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.158797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.159071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.159785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:17.160038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:17.161424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:17.166424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 495/986] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:17.173037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:17.281837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c5184-01d0-40d9-84c9-ef7344d60827 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:17.316691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:17.317295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.317572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.317841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.318464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:17.318763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:17.320156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:17.325532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 492/987] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:17.331179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:17.437720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b42261-eaa0-49c4-a33b-ef0113028fb0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:17.463056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:17.472513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:17.473117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.473393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.473661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.474363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] RBAC: Adding query filter params (interface=admin, service_id=4b14f725fe5e4b22bb799c853db67a32, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:17.474738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:17.476129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bbb1f-a276-4d53-a77c-3acc1c65c401 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:17.486103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 496/988] 10.222.0.46 () {56 vars in 1249 bytes} [Fri Sep 9 21:13:17 2022] GET /identity/v3/endpoints?service_id=4b14f725fe5e4b22bb799c853db67a32&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:17.487743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:17.492511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:17.590205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d33458-367c-4d3c-931f-c1c13d34910f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:17.623858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:17.627218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.627218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.627218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.627218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:17.627218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca732-2e0f-45de-a397-f0981f620d81 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:17.647598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 493/989] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:17 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:13:17.846678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971939f-9661-4530-b823-bdad56060511 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.846966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971939f-9661-4530-b823-bdad56060511 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.847149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971939f-9661-4530-b823-bdad56060511 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.848650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 497/990] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:17.853666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:17.854105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:17.854194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:17.873311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:17.874616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:17.938032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:17.944233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 494/991] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:18.212436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971939f-9661-4530-b823-bdad56060511 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:18.315613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971939f-9661-4530-b823-bdad56060511 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:18.348554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:18.349197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:18.349437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:18.349703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:18.350274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:18.351658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:18.360975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:18.361994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:18.362847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:18.364377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 498/992] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 21:13:18 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:18.367865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:18.466788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784d22e-0d85-4e05-8c07-b5d258ef4f5e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:18.500691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:18.501294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:18.501569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:18.501836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:18.502456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:18.502751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:18.504126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ea79a-ab1f-4594-b27b-8ff2ed72e4c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:18.508936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 495/993] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:18.514374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:18.573045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:18.620195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb71ac-e2b3-4e45-a9b3-1129861d52ba None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:18.660553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:18.660553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:18.660553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:18.661539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:18.662165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:18.662460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:18.663846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b7b12-a390-493d-8ff7-9dfcc3eb85aa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:18.675569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 499/994] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:19.104293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dedc7-4cf5-493a-aa77-d0e9a4193ad6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.104875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dedc7-4cf5-493a-aa77-d0e9a4193ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.104875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dedc7-4cf5-493a-aa77-d0e9a4193ad6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.106419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 496/995] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:19.111800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.112092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.112372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.131392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:19.132616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:19.201443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 500/996] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:19.471972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dedc7-4cf5-493a-aa77-d0e9a4193ad6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:19.535556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:19.578914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dedc7-4cf5-493a-aa77-d0e9a4193ad6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:19.613321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:19.613996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.614279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.614550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.615125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:19.616507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:19.626830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:19.632117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:19.632995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 497/997] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:19.638024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:19.688023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:19.746838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b5b1c-7a5a-480a-be3b-943ac7eea796 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:19.780819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:19.781428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.781702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.781970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.782603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:19.782900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:19.784283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead0be2-fe97-4cc4-9eff-0a2683ee2c24 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:19.794207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 501/998] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:19.795796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:19.831962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f7f5-9948-43a3-8560-e8f779e73f52 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.832303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f7f5-9948-43a3-8560-e8f779e73f52 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.832517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f7f5-9948-43a3-8560-e8f779e73f52 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.833984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 502/999] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:19.838641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.839567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.839851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.857580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:19.858762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:19.902647 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996efc4-7a6f-4bf8-a36d-b3d7b509fba5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:19.932039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:19.933341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 503/1000] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:19.944036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:19.944641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:19.944938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:19.945208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:19.945833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:19.946128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:19.947474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:19.957144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 498/1001] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:19.958727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.002036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:20.059575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93f82f-5ca1-40bf-b382-a93e612d6e14 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.093488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.094087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.094367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.094639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.095329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] RBAC: Adding query filter params (interface=public, service_id=178036d25b874f64a6c0a00920b954e7, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:20.095659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.097015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.102237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 504/1002] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:13:19 2022] GET /identity/v3/endpoints?service_id=178036d25b874f64a6c0a00920b954e7&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:20.108297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.207294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968aac7-45f8-407d-8b71-ecdc98a31d41 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.218084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.245157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.245770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.246049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.246319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.247089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.249093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.256490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 499/1003] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:20 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:13:20.327315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93b13-e930-42fe-891a-c5c7fb9085f3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.336377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:20.367976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.368615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.368890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.369157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.369743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.371526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:20.381131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:20.382197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:20.383779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 505/1004] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 21:13:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:20.388959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.454819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:20.503841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ea552-e35e-486f-ab6c-20cc91013200 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.540880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.541413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.541693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.541990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.543058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:20.543367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.544793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.554223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 500/1005] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:20.555471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.667887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901a01-ae83-48ae-b17c-d21984ba3c5a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.703398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.704048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.704329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.704601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.705232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:20.705534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.706903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86b772-904b-498e-8ef3-f72cc897e5fb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.712099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 506/1006] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 21:13:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:20.718023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:20.826350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f23f-27a4-4282-9c3a-55a02064030c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:20.843665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:20.846203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52c86-aa33-4edd-9bb2-b2a053ef6bdd None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.846737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52c86-aa33-4edd-9bb2-b2a053ef6bdd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.847119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52c86-aa33-4edd-9bb2-b2a053ef6bdd None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.848826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 507/1007] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:20.855073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9742-787c-4650-8733-662993e534ab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.855401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9742-787c-4650-8733-662993e534ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.855796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9742-787c-4650-8733-662993e534ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.863196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:20.863975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:20.864353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:20.864725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:20.865494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:20.866946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:20.874693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c8df9742-787c-4650-8733-662993e534ab None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:20.876054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8df9742-787c-4650-8733-662993e534ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:20.891572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 501/1008] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:20.915000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:20.969602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 508/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:21.240935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:21.341440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a0e5a-acbb-46fc-a9de-feb97327c7eb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:21.375283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:21.375910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:21.376188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:21.376455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:21.377094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:21.378448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6319f-5505-4e9a-bf52-c3c4e5c93bf7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:21.395344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 502/1010] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:21 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:22.461568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb4be-9fc5-4fd2-8366-59769a708776 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:22.462134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb4be-9fc5-4fd2-8366-59769a708776 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:22.462134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb4be-9fc5-4fd2-8366-59769a708776 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:22.463495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 509/1011] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:22.468704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:22.469145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:22.469290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:22.487797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:22.489003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:22.520424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:22.567565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 503/1012] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:22.833381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdb4be-9fc5-4fd2-8366-59769a708776 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:22.938003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdb4be-9fc5-4fd2-8366-59769a708776 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:22.973107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:22.973758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:22.974035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:22.974304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:22.974905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:22.976346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c22f25a3-7de8-4bba-b846-ec8ebf40f4ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:22.986348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:22.987476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:22.992522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 510/1013] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:22.994317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:23.099309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250f4c-3a75-4666-bb7a-9c72c9826ca8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.099493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250f4c-3a75-4666-bb7a-9c72c9826ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.099806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250f4c-3a75-4666-bb7a-9c72c9826ca8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.101159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 511/1014] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:23.103945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948743f-0c5c-4999-afc0-da23564fddb9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:23.106463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.106751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.107028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.125597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:23.126792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:23.142415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:23.143030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.143309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.143613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.144249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:23.144550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:23.145603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:23.145906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:23.151702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 504/1015] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:23.157299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:23.186641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:23.206791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 512/1016] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:23 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:23.267523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eebf8-4c11-4e79-82ec-61fef21d9362 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:23.301816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:23.302419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.302696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.302965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.303613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:23.303914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:23.305263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:23.315583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 505/1017] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 21:13:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:23.316552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:23.416362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c31bae-20e5-4eb9-9c05-451c37b23466 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:23.451511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:23.452136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.452413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.452686 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.453362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:23.454731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc0dee-2024-4c28-85bd-88644211ac6b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:23.475502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 513/1018] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:23.476610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:23.587606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:23.598879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637abb97-3420-4f0b-ac7b-5e47dab135a5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:23.603567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4045f-4c3a-43d7-94d4-c5fbaf781d14 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.603847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4045f-4c3a-43d7-94d4-c5fbaf781d14 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.604125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4045f-4c3a-43d7-94d4-c5fbaf781d14 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.606031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 514/1019] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:23.611478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.611796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.612077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.634939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:23.637379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:23.641398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:23.651691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:23.652339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.652625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.652896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.654020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:23.655450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:23.657225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 506/1020] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:23.662887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:23.717791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 515/1021] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:23.780247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31063092-ae87-410c-99ad-90de7386c3b7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:23.817978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:23.818596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:23.818874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:23.819150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:23.819832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:23.821261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:23.855582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 507/1022] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:23 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:24.004875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:24.122680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e064c7-b3b7-4ff8-aad6-d3ae3832cab7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:24.163708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:24.164358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.164924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.165199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.165794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:24.167773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:24.177242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:24.178344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:24.179679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 516/1023] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 21:13:24 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:24.185619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:24.199200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:24.299992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828778-dda7-4e61-8aff-8398db07c414 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:24.335526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:24.336220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.336505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.336778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.337416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] RBAC: Adding query filter params (name=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:24.337720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:24.340288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:24.350352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 508/1024] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:24.351994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:24.356498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:24.466559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165dee5-3933-45b6-8744-c50de019561f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:24.504702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:24.505342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.505637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.505917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.506612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:24.506921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:24.508329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:24.518515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 517/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:24.520139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:24.597081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:24.643050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa257b3-bdfd-4663-a258-7ffb4efac12a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:24.682233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:24.683211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.683737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.684244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.685190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] RBAC: Adding query filter params (interface=public, service_id=12eea46647e642c686261400a03b34eb, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:24.685771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:24.687352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d57dd-7396-4507-b1cb-d3954b49026e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:24.693189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 509/1026] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:13:24 2022] GET /identity/v3/endpoints?service_id=12eea46647e642c686261400a03b34eb&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:24.699404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:24.772366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:24.810846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084ddc-1bf0-4dac-ba37-afb5825322a5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:24.849712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:24.850246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:24.850521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:24.850788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:24.851398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:24.852824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80dfc7f-8adb-4a9e-aa2d-7b8b67dbda14 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:24.867583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 518/1027] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:13:24 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:13:26.012751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c082-6d8d-4363-878b-00ed2af434b5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.013276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c082-6d8d-4363-878b-00ed2af434b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.013276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c082-6d8d-4363-878b-00ed2af434b5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.014778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 510/1028] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:26.019979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc44758-a022-4022-b524-e6d110aa7076 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.020402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc44758-a022-4022-b524-e6d110aa7076 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.020567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc44758-a022-4022-b524-e6d110aa7076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.041137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cdc44758-a022-4022-b524-e6d110aa7076 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:26.043235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cdc44758-a022-4022-b524-e6d110aa7076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:26.116263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 519/1029] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:26.152274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538c4cd-abec-4981-9442-72154628c3f4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.152690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538c4cd-abec-4981-9442-72154628c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.152856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538c4cd-abec-4981-9442-72154628c3f4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.154285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 511/1030] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:26.159301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.159613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.159886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.178759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:26.179949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:26.238409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:26.250539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 520/1031] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:26.382950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538c4cd-abec-4981-9442-72154628c3f4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:26.485723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538c4cd-abec-4981-9442-72154628c3f4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:26.518634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:26.520175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:26.520812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.521091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.521364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.524099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:26.538634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:26.544518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:26.545481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:26.546331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:26.547837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 512/1032] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:26.552658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:26.632507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6c3c6-2555-4878-90ea-11183c8aea26 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:26.668680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4192-eb7a-4ed0-a828-de587e3d9219 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:26.670619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:26.671954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.671954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.671954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.672943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:26.674486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:26.680017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 521/1033] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:26.681513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:26.710761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:26.711362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.711764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.712018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.712601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:26.712897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:26.714779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:26.722949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 513/1034] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:26.731095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:26.750230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:26.795325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:26.804153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba5e8c-fb85-4460-af04-0a5f6a4af379 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:26.838242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:26.839582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.839872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.840138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.840661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7bc57-2069-46a6-8b90-ce57be9930cf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:26.840778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:26.842181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:26.871573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 522/1035] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:13:26 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:26.883884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:26.884665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:26.885009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:26.885419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:26.898635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:26.899996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-01833acd-8fe8-46e9-9782-cdafeeb4f131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:26.918211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:26.919350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:26.920335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:26.920335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:26.920335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:26.920335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 514/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:26.926060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.042695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef680b-5bc8-4419-914c-f6bf47c1fe82 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.054756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430cef-4332-473e-90b7-040087e86a31 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.055273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430cef-4332-473e-90b7-040087e86a31 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.055273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430cef-4332-473e-90b7-040087e86a31 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.056628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 515/1037] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:27.063279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.063565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.063833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.063934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.084971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:27.086149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:27.087882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.088529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.088808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.089081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.089736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:27.090040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.091411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.105225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 523/1038] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:27.110772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.170649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 516/1039] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:27.219299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b996-c66a-43b9-8ee0-bb318f06076b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.254977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.255630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.255911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.256182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.263560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.264999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:27.277261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:27.278244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:27.279228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:27.280361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 524/1040] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:27.284496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.331571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.390816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbc348-4a34-4f09-a1e3-a4ccd9798cbc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.425939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.426557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.426899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.427264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.428139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:27.428511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.429988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.438597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.455130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 517/1041] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:27.456275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.482938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.554111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634b537-24ed-47b1-96ee-285038f8e5ed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.566160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.571853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af12b0-7f68-4238-b61c-820889fbf90e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.591492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.592160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.592444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.592917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.593306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.594696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:27.603733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:27.605513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 21:13:27.606528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:27.610526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 525/1042] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:27.611370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.613939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.614833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.615211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.616392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.691925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.693357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.723795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.727762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770f5ff-bd54-482c-b6cd-5079230d0701 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.766020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.766724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.767087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.767429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.768158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:27.768463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.769829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.774760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 526/1043] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:27.781171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.798898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.807926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 518/1044] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:27.810268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.891830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca473ba-6d32-4832-866c-105468c554dd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.924901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff207-d8cb-48f5-9839-67ccbb7f164b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:27.927128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.927760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.928038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.928307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.928931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:27.929232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.930595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.935454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 527/1045] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:27.941301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:27.967841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:27.968411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:27.968697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:27.968968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:27.971658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:27.979854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:27.981287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:27.996667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:27.997652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:27.998646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:27.998646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:27.998646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:27.998646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 519/1046] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:28.003301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:28.049000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966bfc8-4d10-48f3-b718-9d61406f5d58 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:28.056556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:28.086687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:28.087294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:28.087991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:28.088333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:28.089897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:28.091676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:28.114108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947762b-d2ab-4e17-8c7f-e7297b24f534 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:28.115875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 528/1047] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:13:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:13:28.172525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:28.173237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:28.173521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:28.173789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:28.174479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:28.174781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:28.176244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:28.195176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 520/1048] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:28.196623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:28.293596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:28.313068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a88ff9-cc2e-486c-87bf-f4ddc155978d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:28.348443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763163e-637d-467b-9531-cd60054d414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:28.349076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763163e-637d-467b-9531-cd60054d414e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:28.349357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763163e-637d-467b-9531-cd60054d414e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:28.349633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763163e-637d-467b-9531-cd60054d414e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:28.356549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763163e-637d-467b-9531-cd60054d414e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:28.358031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763163e-637d-467b-9531-cd60054d414e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d763163e-637d-467b-9531-cd60054d414e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:28.369415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:28.371275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:28.372221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 529/1049] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:28.377214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:28.479838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ade1-af2f-4b4e-a679-0f6ec00ffe84 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:28.487358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:28.520183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:28.520793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:28.521070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:28.521339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:28.522026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:28.522327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:28.523763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3ac8-da83-46a5-8962-06382ba1cff6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:28.541013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 521/1050] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:28.543402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763163e-637d-467b-9531-cd60054d414e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:28.655330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763163e-637d-467b-9531-cd60054d414e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:28.691464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:28.692415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:28.692847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:28.693289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:28.694223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] RBAC: Adding query filter params (role.id=338b17a9779a44a38b33c0ca5107a2f3, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:28.694738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:28.696485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d57d-5755-4a5e-9501-64b1a1ee58d3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:28.707738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 530/1051] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:28 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=338b17a9779a44a38b33c0ca5107a2f3 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:29.131810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647cc13-1a8a-4149-aad4-afbb77a3704b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:29.132344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647cc13-1a8a-4149-aad4-afbb77a3704b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:29.132344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647cc13-1a8a-4149-aad4-afbb77a3704b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:29.133928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 522/1052] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:29.139236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:29.139645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:29.139812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:29.158938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:29.160130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:29.162119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:29.229990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 531/1053] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:29.500717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647cc13-1a8a-4149-aad4-afbb77a3704b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:29.605985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647cc13-1a8a-4149-aad4-afbb77a3704b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:29.648429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:29.649051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:29.649328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:29.649597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:29.660322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:29.665064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:29.666005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:29.667488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 523/1054] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:29.672825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:29.769757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8570df-9896-46d2-9015-598c0eebdec3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:29.805119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:29.805724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:29.806007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:29.806276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:29.807983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:29.808292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:29.809734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:29.815964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:29.825661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 532/1055] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:29.827088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:29.854621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:29.928439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba29915-09e9-4af9-9373-f78a5ce0c60d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:29.964072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:29.964702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:29.964976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:29.965244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] PATH_INFO: `/v3/users/b872f3f447874e18b51252f3e0732074` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:29.978575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b872f3f447874e18b51252f3e0732074)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:29.980169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:29.986472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 524/1056] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:29 2022] GET /identity/v3/users/b872f3f447874e18b51252f3e0732074 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:29.987670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:30.088442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34376-6999-4e2f-8c95-b5b79bfdd48f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:30.122282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:30.122880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.123199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.123482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.192044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:30.195672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:30.197382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef329a10-c97e-428d-bf76-f741b68e9f4d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:30.307511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 533/1057] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:30.308924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:30.374348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4fcd5-b98d-4de9-95dd-bf296244f50e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.374544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4fcd5-b98d-4de9-95dd-bf296244f50e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.374806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4fcd5-b98d-4de9-95dd-bf296244f50e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.376325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 534/1058] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:30.381220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.381506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.381784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.393796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:30.400451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:30.402239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:30.415282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7bda9-a33e-4c17-80eb-06b8bdc7e6c8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:30.454531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:30.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.455431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.455725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.456831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:30.458231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:30.459885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 525/1059] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:30 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:30.467851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:30.488953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 535/1060] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:30.575771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf0a2b-d746-45cf-91c7-4d40025c9157 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:30.608554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:30.609146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.609422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.609692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.610409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9, user.id=b872f3f447874e18b51252f3e0732074) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:30.610714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:30.612152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fc3be-04cb-4b41-a1c5-0f1fd90a9f06 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:30.623571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 526/1061] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:30 2022] GET /identity/v3/role_assignments?user.id=b872f3f447874e18b51252f3e0732074&scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:30.761525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:30.802909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:30.873137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eb247-efd1-45ae-937d-25425dd8d62d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:30.908659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:30.909173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.909452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.909720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.912388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8084-2f4b-46ce-9fbf-22913328a776 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.912687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8084-2f4b-46ce-9fbf-22913328a776 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.912962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8084-2f4b-46ce-9fbf-22913328a776 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.915282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 527/1062] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:30.920826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:30.921396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:30.921396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:30.939688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:30.940935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:30.956609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:30.985835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:30.987291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.019516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 528/1063] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:31.026661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:31.111442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 536/1064] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:31.112699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.215229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ce3e2-3a41-4433-914c-81398fb51150 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.250986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.251644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.251929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.252204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.262338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:31.263794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.270507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:31.281165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:31.282221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:31.288421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:31.288421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:31.288421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:31.288421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 529/1065] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:31.288696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.291184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.396861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa87d9-969b-4970-8e8c-7e24aab6b73c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.398358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c3af-135a-4c01-aeb0-5cc074cd9ae1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.427350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:31.434653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.435260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.435591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.435840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.436203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.436486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:31.436788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:31.436893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.437135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.437348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.438156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.448568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:31.453932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:31.454933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:31.456427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:31.457785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 537/1066] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:31.459916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 530/1067] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:31.461219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.580039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c7f71-ae52-4f30-b508-1ab4a8e76eda None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.616404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.617025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.617305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.617580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.618247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:31.618552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:31.620067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.628971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 538/1068] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:31.636402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.696812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:31.742704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fb4b7-3ab0-4b1a-9b33-f329066cc136 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.780938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.781570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.781877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.782148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.793237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:31.794674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:31.808677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:31.809779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:31.810678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:31.810678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:31.810678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:31.810678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 531/1069] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:31.815918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:31.871980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:31.922100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb461b93-49ea-4373-a9ed-51c938d35545 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:31.954151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:31.954747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:31.955024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:31.955294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:31.955960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:31.956261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:31.957618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:31.969509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 539/1070] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:31.975185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.071459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986c08b-8afb-4247-bf80-eddf7c63ad7b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.081442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:32.105357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.105987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.106266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.106539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.113852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.115293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:32.126045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:32.127143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:32.128701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 532/1071] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:13:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:32.132769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.234681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ad060-1948-4ced-9a47-0f2c6f8286f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.247509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:32.269125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.269724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.270002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.270276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.270997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:32.271297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.272760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.291662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 540/1072] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:13:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:32.297155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.394003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9de65-1707-42aa-88c6-1f89e7cb2f00 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.427849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.428503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.428736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.429007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.439554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.440985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:32.455161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:32.456261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:32.457181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:32.457181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:32.457181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:32.457181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 533/1073] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:32.462045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.558766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb72b39-9238-4f14-b14c-a1a27018a6c3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.592018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.592615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.592893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.593162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.593797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:32.594094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.595445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.611022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 541/1074] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:32.612313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.639174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:32.717622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd7cf-a45d-431c-96de-8ffb4cca5e55 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.750242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.750861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.751141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.751412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.757987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.759414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-df36da8b-e467-4ad3-8a69-133da5606518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:32.771011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:32.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:32.772946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 534/1075] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:32.779677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.781341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0d296-5f92-4053-9046-57b2039a37d8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.781626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0d296-5f92-4053-9046-57b2039a37d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.781900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0d296-5f92-4053-9046-57b2039a37d8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.783235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 535/1076] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:13:32.788194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.788480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.788757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.806486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:32.808358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:32.810411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:32.876191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 536/1077] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:32.879355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d031f1-d3c1-4982-8dce-1124ebbec9f4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:32.912428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:32.913027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:32.913307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:32.913579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:32.914290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:32.914594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:32.916096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:32.927631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 542/1078] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:32.934518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:32.949047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:33.031316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd382c1b-99be-451a-ad55-24670e61bffd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.064657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.065296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.065569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.065837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/338b17a9779a44a38b33c0ca5107a2f3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.067980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=dac985d2461d4132af309ded8c7ce1a1, role_id=338b17a9779a44a38b33c0ca5107a2f3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:33.069391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:33.082182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 537/1079] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:13:32 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/dac985d2461d4132af309ded8c7ce1a1/roles/338b17a9779a44a38b33c0ca5107a2f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:33.145351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:33.240173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:33.254143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9d42e-6b6e-4ad0-b275-44993b50a4d5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.290108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.290661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.290938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.291340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.301639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:33.306426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:33.307436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:33.308413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 543/1080] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:33.313959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:33.387982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:33.417964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d11145-c9b3-4d6f-9f27-29bd330b229a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.452038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.452696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.452899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.453167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.453822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:33.454118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:33.455461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:33.463393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 538/1081] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:33.469143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:33.570410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805463b-3636-4e44-8179-7dde6f17697a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.605079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.605707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.605989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.606265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] PATH_INFO: `/v3/users/b872f3f447874e18b51252f3e0732074` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.607391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] RBAC: Authorizing `identity:get_user(user_id=b872f3f447874e18b51252f3e0732074)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:33.608860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:33.610445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 544/1082] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:33 2022] GET /identity/v3/users/b872f3f447874e18b51252f3e0732074 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:33.616127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:33.716173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52e082-fd15-4254-82a2-ae61ffbfae13 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.748562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.749182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.749458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.749731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.750843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:33.752294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f3405-9b15-4b2b-89b1-a88633bd6d1c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:33.753923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 539/1083] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:33 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:33.759692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:33.778984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:33.859549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094365d-28ff-47b2-8e96-5877df9edfb7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:33.893443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:33.894086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:33.894363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:33.894634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9/users/b872f3f447874e18b51252f3e0732074/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:33.896533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] RBAC: Authorizing `identity:create_grant(project_id=e58b16c9ddab41e99bcb41054f5d62f9, user_id=b872f3f447874e18b51252f3e0732074, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:33.897956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc9249-57b1-415b-acb4-00ddcdbc690a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:33.910713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 545/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:13:33 2022] PUT /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9/users/b872f3f447874e18b51252f3e0732074/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:35.249530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9c22-65b7-45cb-b247-688e7c76991a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:35.250088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9c22-65b7-45cb-b247-688e7c76991a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:35.250088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc9c22-65b7-45cb-b247-688e7c76991a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:35.251390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 540/1085] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:35.256687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:35.257098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:35.257205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:35.276282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:35.277449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:35.341823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 546/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:35.615161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc9c22-65b7-45cb-b247-688e7c76991a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:35.618082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:35.719763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc9c22-65b7-45cb-b247-688e7c76991a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:35.753330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:35.753951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:35.754232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:35.754505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:35.765107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:35.769898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:35.770948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:35.771857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 541/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 21:13:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:35.777538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:35.787775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:35.877102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e2f54-53fa-44e6-9497-0175f5bd9d4b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:35.910791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:35.911392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:35.911703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:35.911975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:35.912638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:35.912936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:35.914309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f45321-e296-4c8d-aae9-a7e4cd2c6d9d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:35.922063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 547/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:13:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:35.928012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:35.994699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.033221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edfa77-344d-4fc5-8a70-12b9bae3f0b8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.053541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea938-2cea-4613-bbc2-4d04f0eb684c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.053826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea938-2cea-4613-bbc2-4d04f0eb684c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.054098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea938-2cea-4613-bbc2-4d04f0eb684c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.055425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 548/1089] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:36.060803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.061078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.061343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.067063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.067744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.068024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.068294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.079025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.079655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:36.080486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.080838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:36.095812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:36.096814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:36.097716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:36.097716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:36.097716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:36.097716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 542/1090] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:36.102466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.142495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.151823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 549/1091] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:36.202657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ad33-61c9-498f-b1f0-59240adafcab None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.216145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.235397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.236018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.236292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.236558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.237188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:36.237490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.238821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.250291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 543/1092] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:36.255908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.353107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937cbc5-8196-4e81-ae6a-4f7b8b38f36a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.386398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.387041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.387316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.387608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.394428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.395824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:13:36.407175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:36.408199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 21:13:36.409206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:36.411830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 550/1093] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:36.413914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.416567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.475294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.513413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64905-fdcd-477c-88b7-501f185e892e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.517274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58632b-64db-44fe-b077-f45d06bdefaf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.546478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.547082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.547364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.547661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.548391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:36.548698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.550116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.550443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.551063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.551340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.551644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.563663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:36.567501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 544/1094] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:36.568416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:36.569355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:13:36.570218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 551/1095] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:36.573994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.575968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.604207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.673010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe25a-ed2e-4b9e-86e0-1df4653cb13a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.676851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e582a-f64d-4c5d-9b56-694a6a5e61a8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.696416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.706907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.707512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.707830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.708169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.711418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.712052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.712331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.712607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.713291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:36.713593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.714966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.723437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 552/1096] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:36.729821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.772705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.774124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.803229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.829866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9207-1b80-4c5e-88c4-af1e4b3aa98d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.865262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:36.865879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:36.866155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:36.866430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] PATH_INFO: `/v3/users/b872f3f447874e18b51252f3e0732074` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:36.867646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] RBAC: Authorizing `identity:get_user(user_id=b872f3f447874e18b51252f3e0732074)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:36.869696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:36.871253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 553/1097] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/users/b872f3f447874e18b51252f3e0732074 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:36.875772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 545/1098] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:36.876894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.881989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:36.926213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:36.975726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8fa6-bd3a-4985-8f62-184d07dbb338 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:36.981724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8083788-9631-41e2-b01d-c2c98f080128 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.009802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.010338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.010459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.010684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.010963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.015564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.016180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.016453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.016719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.026645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.028119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:13:37.042675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:37.043791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:13:37.044738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:13:37.044738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:13:37.044738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:37.044738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 546/1099] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:37.049464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.078612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.080097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.127628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.146160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeadc9c-f487-4b86-8c53-d426d93c4a06 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.179981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 554/1100] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:37.181193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.181881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.182243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.182513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.183150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:37.183448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.184817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.186145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.196789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 547/1101] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:37.202081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.246778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.283808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8325-6500-443b-b09a-aa98c3e6b020 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.302075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967fa87-044e-41b8-ab5b-5097898a7ec8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.317991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.318614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.318892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.319169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.320297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.321713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.323445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 555/1102] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:37.329669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.335858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.336488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.336766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.337036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.344473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.345895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.347980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:13:37.358314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:37.359491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:37.360728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 548/1103] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:13:37.365052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.435843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb974f64-abc7-4991-92ef-b2b3f6df5f2a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.469893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.470491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.470771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.471047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.471799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9, user.id=b872f3f447874e18b51252f3e0732074) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:37.472104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.473857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.477078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3405-677a-418b-b654-ef836080d4be None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.479768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.480618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 556/1104] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/role_assignments?user.id=b872f3f447874e18b51252f3e0732074&scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:37.518512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.519135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.519412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.519721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.520442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:37.520730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.522191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d31f7-c25b-43a0-a4e3-3a1a2e11732d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.537085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 549/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:13:37.544236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:37.641645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:37.661208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa443c-d461-4b9c-a256-8fbd79adb80e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:37.694914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:37.695454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:37.695796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:37.696076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:37.696806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] RBAC: Adding query filter params (role.id=338b17a9779a44a38b33c0ca5107a2f3, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=dac985d2461d4132af309ded8c7ce1a1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:37.697703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:37.699186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d7abf-43bf-4b8c-9c49-dd517b8fdda7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:37.705021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 557/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:37 2022] GET /identity/v3/role_assignments?user.id=dac985d2461d4132af309ded8c7ce1a1&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=338b17a9779a44a38b33c0ca5107a2f3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:39.666992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ccdf-4b30-4696-a28a-f2ae8a9356aa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:39.667478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ccdf-4b30-4696-a28a-f2ae8a9356aa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:39.667592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030ccdf-4b30-4696-a28a-f2ae8a9356aa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:39.668879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 550/1107] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:39.674089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:39.674528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:39.674622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:39.693020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:39.694201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:39.758954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 558/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:40.038594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030ccdf-4b30-4696-a28a-f2ae8a9356aa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:40.137469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030ccdf-4b30-4696-a28a-f2ae8a9356aa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:40.170841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:40.171448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:40.171756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:40.172028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:40.173118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:40.174530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b31c5-b034-48a9-98bd-54502dcac80c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:40.176091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 551/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:40.181715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:40.254731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:40.281447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023c345-487a-4ad4-8b92-a9aab0c9caf1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:40.316726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97665469-c507-4897-91fc-9085358298a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:40.317244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665469-c507-4897-91fc-9085358298a5 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:40.317525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665469-c507-4897-91fc-9085358298a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:40.317794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97665469-c507-4897-91fc-9085358298a5 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:40.318434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97665469-c507-4897-91fc-9085358298a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:40.319875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97665469-c507-4897-91fc-9085358298a5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:40.340003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 559/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:13:40 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:42.515316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f82f3-ccc2-4ac1-b7bf-8c987f207ef5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:42.515881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f82f3-ccc2-4ac1-b7bf-8c987f207ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:42.515881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f82f3-ccc2-4ac1-b7bf-8c987f207ef5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:42.517237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 552/1111] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:42.522461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:42.522903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:42.523004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:42.542712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:42.543948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:42.608411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 560/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:42.882618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f82f3-ccc2-4ac1-b7bf-8c987f207ef5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:42.917022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:42.987768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f82f3-ccc2-4ac1-b7bf-8c987f207ef5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:43.019706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:43.020315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:43.020588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:43.020855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] PATH_INFO: `/v3/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:43.022265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] RBAC: Authorizing `identity:get_role(role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:43.023629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:43.025332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 553/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:42 2022] GET /identity/v3/roles/da0177264e704fa19ac1889c21e21eb5 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:43.031136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:43.090269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:43.129036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09f779-1c9e-46dd-a711-318cf9546dc5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:43.162317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:43.162928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:43.163203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:43.163471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] PATH_INFO: `/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:43.176494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] RBAC: Authorizing `identity:get_user(user_id=d40bfc2e5ea94c5d8727b4e51a7668fd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:43.177906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:43.179475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 561/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:43 2022] GET /identity/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:43.185711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:43.281642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:43.284657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f8a8-81c0-4ce1-8963-af6da760d6cb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:43.317483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:43.318073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:43.318348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:43.318616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:43.382152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:43.383583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:43.473052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:43.477082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 554/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:43.483852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:43.583725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e84fbf-7dc2-4dbf-9aa0-63734eb75333 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:43.603778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:43.617411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:43.618025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:43.618301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:43.618635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:43.619751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:43.621161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9f55b-c0a5-41e6-a584-36ddec8ce9df None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:43.622756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 562/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:43 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:43.629208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:43.726270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcac19-5edf-4ece-b15a-33f8c73e0e91 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:43.758316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:43.758908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:43.759780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:43.760054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:43.760857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9, user.id=d40bfc2e5ea94c5d8727b4e51a7668fd) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:43.761162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:43.762590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe392a7-0d9d-41df-9b6e-30c8fc6a3555 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:43.768747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 555/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:43 2022] GET /identity/v3/role_assignments?user.id=d40bfc2e5ea94c5d8727b4e51a7668fd&scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:45.880353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1a29-3ea7-4071-84d2-07ef7946ef85 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:45.880875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1a29-3ea7-4071-84d2-07ef7946ef85 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:45.880875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1a29-3ea7-4071-84d2-07ef7946ef85 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:45.882216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 563/1118] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:45.887613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:45.888036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:45.888130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:45.907013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:45.908222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:45.963893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:45.971814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 556/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:46.240695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1a29-3ea7-4071-84d2-07ef7946ef85 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.339813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1a29-3ea7-4071-84d2-07ef7946ef85 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.373268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.373882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.374162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.374437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] PATH_INFO: `/v3/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.375855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] RBAC: Authorizing `identity:get_role(role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.377218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.378505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 564/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/roles/da0177264e704fa19ac1889c21e21eb5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:46.385035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.479776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9920a-22e1-4b2b-8f2c-b85dfeeefc74 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.513037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.513580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.513853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.514116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] PATH_INFO: `/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.515204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] RBAC: Authorizing `identity:get_user(user_id=d40bfc2e5ea94c5d8727b4e51a7668fd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.516632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.518168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 557/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:46.523961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.579812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:46.622815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c076e92-472b-4f26-a45b-a5a6209f67b9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.656298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.656920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.657198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.657469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.658571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.660097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76f2b8-7b55-4578-a272-076530869718 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.661708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 565/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:46 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:46.668182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:46.726299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:46.763598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ea8-bdd5-4d22-a09d-8ea67b62839c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:46.795879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:46.796510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:46.796784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:46.797054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9/users/d40bfc2e5ea94c5d8727b4e51a7668fd/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:46.799128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] RBAC: Authorizing `identity:create_grant(project_id=e58b16c9ddab41e99bcb41054f5d62f9, user_id=d40bfc2e5ea94c5d8727b4e51a7668fd, role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:46.800565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249321b-0927-4c5a-a3c3-727d09b4aa05 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:46.813517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 558/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:13:46 2022] PUT /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9/users/d40bfc2e5ea94c5d8727b4e51a7668fd/roles/da0177264e704fa19ac1889c21e21eb5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:13:48.923778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f921de-2d34-41c3-a115-3d4f7c1216dc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:48.924322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f921de-2d34-41c3-a115-3d4f7c1216dc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:48.924322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f921de-2d34-41c3-a115-3d4f7c1216dc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:48.925648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 566/1124] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:48.930981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:48.931605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:48.931908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:48.950122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:48.951309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:49.016016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 559/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:49.286506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f921de-2d34-41c3-a115-3d4f7c1216dc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:49.385452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f921de-2d34-41c3-a115-3d4f7c1216dc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:49.407959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:49.418948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:49.419597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:49.419882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:49.420159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] PATH_INFO: `/v3/roles/da0177264e704fa19ac1889c21e21eb5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:49.421541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] RBAC: Authorizing `identity:get_role(role_id=da0177264e704fa19ac1889c21e21eb5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:49.422913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:49.424264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 567/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:49 2022] GET /identity/v3/roles/da0177264e704fa19ac1889c21e21eb5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:49.429896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:49.480515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:49.529066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2fa2e-489d-4c48-bbc7-ac1728a964cf None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:49.562308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:49.562870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:49.563149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:49.563423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] PATH_INFO: `/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:49.564566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] RBAC: Authorizing `identity:get_user(user_id=d40bfc2e5ea94c5d8727b4e51a7668fd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:49.565986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:49.567579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 560/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:49 2022] GET /identity/v3/users/d40bfc2e5ea94c5d8727b4e51a7668fd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:49.573707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:49.672223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4250-8ef6-483d-bed4-beaee98fae5a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:49.704942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:49.705577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:49.705849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:49.706122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:49.762826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:49.772551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:49.773996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:49.869194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 568/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:49.875039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:49.971094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6dfab-d4b7-45f2-9848-b084039e7e85 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:49.990517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:50.004059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:50.004677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:50.004947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:50.005213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] PATH_INFO: `/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:50.006305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] RBAC: Authorizing `identity:get_project(project_id=e58b16c9ddab41e99bcb41054f5d62f9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:50.007738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff743b-838d-46ba-80b6-5ed9d5644c7b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:50.009607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 561/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:49 2022] GET /identity/v3/projects/e58b16c9ddab41e99bcb41054f5d62f9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:50.015752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:50.103457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:50.112104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8e9a-61f0-4fa2-a2a2-7c8a55478b99 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:50.143894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:50.144427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:50.144703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:50.144971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:50.145709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] RBAC: Adding query filter params (role.id=da0177264e704fa19ac1889c21e21eb5, scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9, user.id=d40bfc2e5ea94c5d8727b4e51a7668fd) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:50.146012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:50.147429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f19ce2-e5bd-413e-a61e-eab040dd70cc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:50.153688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 569/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:50 2022] GET /identity/v3/role_assignments?user.id=d40bfc2e5ea94c5d8727b4e51a7668fd&scope.project.id=e58b16c9ddab41e99bcb41054f5d62f9&role.id=da0177264e704fa19ac1889c21e21eb5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:52.310353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adafe-ebe8-44f2-9b9f-46b296adeb5d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:52.310883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adafe-ebe8-44f2-9b9f-46b296adeb5d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:52.310883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adafe-ebe8-44f2-9b9f-46b296adeb5d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:52.312245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 562/1131] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:52.317702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:52.318135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:52.318229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:52.336909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:52.338076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:52.401986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 570/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 21:13:52.669014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6adafe-ebe8-44f2-9b9f-46b296adeb5d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:52.766593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6adafe-ebe8-44f2-9b9f-46b296adeb5d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:52.798927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:52.799555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:52.799832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:52.800095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:52.801175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:52.802571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350992df-f8f7-4430-aa22-3e697137140f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:52.804156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 563/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:52.809893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:52.892024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:52.907345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f757e4-ba4a-4747-9924-167e4c721357 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:52.940373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:52.940974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:52.941258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:52.941533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:52.942184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:52.943966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b953d-2cc1-48d5-893f-2a6466cb64a0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:52.973720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 571/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:13:52 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:55.148844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637aaea-92b3-4d74-abea-d83d963f6414 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:55.149671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637aaea-92b3-4d74-abea-d83d963f6414 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:55.149766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637aaea-92b3-4d74-abea-d83d963f6414 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:55.151061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 564/1135] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:55.156269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:55.156656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:55.156825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:55.174518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:55.175735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:55.240585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 572/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:55.511862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637aaea-92b3-4d74-abea-d83d963f6414 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:55.567764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:55.611614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637aaea-92b3-4d74-abea-d83d963f6414 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:55.645208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:55.645836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:55.646114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:55.646387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:55.647500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:55.650534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803795f-6855-48ef-b7f0-6ccfc0a3c6b3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:55.652846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 565/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:13:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:55.658589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:55.723130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:55.756973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c7fa-a47e-4e75-b71a-acd370b39744 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:55.790860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:55.791393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:55.791700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:55.791974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:55.792620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:55.794044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66e0d4-6d94-4e18-86f7-5d9e1b4b3e64 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:55.814769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 573/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:13:55 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:13:57.976968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d7c8-7f53-477a-a1d6-eaf866f475a3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:57.977565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d7c8-7f53-477a-a1d6-eaf866f475a3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:57.977565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d7c8-7f53-477a-a1d6-eaf866f475a3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:57.978808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 566/1139] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:13:57.984032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:57.984447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:57.984580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:58.002326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:13:58.003506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:13:58.067862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 574/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:13:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:13:58.339026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d7c8-7f53-477a-a1d6-eaf866f475a3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:58.436847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d7c8-7f53-477a-a1d6-eaf866f475a3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:58.459097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:58.470705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:58.471329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:58.471634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:58.471906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:58.482484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:13:58.487292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:13:58.488392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:13:58.489423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 567/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:13:58.494461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:58.547346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:58.596197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3f4a-d5fc-44f9-b870-a7e9680abd7f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:58.628646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:58.629242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:58.629517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:58.629787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:58.630442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:58.630741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:58.632138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:58.640969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 575/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:58.646888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:58.745050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60f6-b734-46fe-bd20-0d98b58b5a26 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:58.778327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:58.778944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:58.779224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:58.779493 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] PATH_INFO: `/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:58.792128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] RBAC: Authorizing `identity:get_user(user_id=385ffb126f2d4aad9b1e96b67d3c22aa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:58.793550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:58.795129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 568/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:13:58 2022] GET /identity/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:58.801077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:58.897130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ddcdd-99b8-4d2c-b46b-795f27e111b5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:58.912418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:58.929540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:58.930790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:58.931069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:58.931337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:59.000655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:59.002102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:59.096533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 576/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:13:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:13:59.102600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:59.145843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:59.200726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf4073-c262-44c6-a689-7c4fb57877db None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:59.232382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:59.232992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:59.233266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:59.233535 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] PATH_INFO: `/v3/projects/08d6081692b1475f8d37f4f0413b3f34` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:59.234617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] RBAC: Authorizing `identity:get_project(project_id=08d6081692b1475f8d37f4f0413b3f34)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:59.236052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2cae-4bba-40f8-b9c5-67cc235414c3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:59.237643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 569/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:13:59 2022] GET /identity/v3/projects/08d6081692b1475f8d37f4f0413b3f34 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:13:59.244382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:13:59.262146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:13:59.343945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929247-c28a-4ea2-950a-c1e3813f68a2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:13:59.377276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:13:59.377873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:13:59.378152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:13:59.378422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:13:59.379138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=08d6081692b1475f8d37f4f0413b3f34, user.id=385ffb126f2d4aad9b1e96b67d3c22aa) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:13:59.379445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:13:59.380889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac5c52-e674-4440-8647-a191e0068b04 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:13:59.386117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 577/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:13:59 2022] GET /identity/v3/role_assignments?user.id=385ffb126f2d4aad9b1e96b67d3c22aa&scope.project.id=08d6081692b1475f8d37f4f0413b3f34&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:01.494687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d522-7451-49ba-b316-93c569fd38eb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.495340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d522-7451-49ba-b316-93c569fd38eb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.495340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d522-7451-49ba-b316-93c569fd38eb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:01.496583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 570/1147] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:01.501771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.502194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.502302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:01.521181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:01.522339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:01.591922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 578/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:01.861334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d522-7451-49ba-b316-93c569fd38eb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:01.960794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d522-7451-49ba-b316-93c569fd38eb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:01.970951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:01.993769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:01.994378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:01.994653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:01.994923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.004468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:14:02.009937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:02.011106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:14:02.012263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 571/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:02.016968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.028437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:02.112281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289291b3-c80f-4aea-b5e2-d62729b363af None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.145439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.146034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.146334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.146604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.147261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:02.147580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.149303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.157006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 579/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:02.162769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.261461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a23801-cfa5-4ca1-a109-942ab3680b57 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.294463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.295071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.295346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.295641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] PATH_INFO: `/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.296743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] RBAC: Authorizing `identity:get_user(user_id=385ffb126f2d4aad9b1e96b67d3c22aa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.298149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.299734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 572/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:14:02 2022] GET /identity/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:02.306192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.372813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:02.403699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b256-e6f6-490d-865e-b9dbe60e0494 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.437688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.438310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.438588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.438858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] PATH_INFO: `/v3/projects/08d6081692b1475f8d37f4f0413b3f34` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.439982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] RBAC: Authorizing `identity:get_project(project_id=08d6081692b1475f8d37f4f0413b3f34)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.441403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810ff36-dbe3-48de-a457-7d9d225f1e73 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.443011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 580/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:14:02 2022] GET /identity/v3/projects/08d6081692b1475f8d37f4f0413b3f34 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:02.449210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:02.487697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:02.545727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c6612-4599-4bcf-abef-8f683ff6616d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:02.579140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:02.579807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:02.580086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:02.580356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] PATH_INFO: `/v3/projects/08d6081692b1475f8d37f4f0413b3f34/users/385ffb126f2d4aad9b1e96b67d3c22aa/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:02.582222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] RBAC: Authorizing `identity:create_grant(project_id=08d6081692b1475f8d37f4f0413b3f34, user_id=385ffb126f2d4aad9b1e96b67d3c22aa, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:02.583664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3d359-49c0-45bf-a456-64092536b54d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:02.597233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 573/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:14:02 2022] PUT /identity/v3/projects/08d6081692b1475f8d37f4f0413b3f34/users/385ffb126f2d4aad9b1e96b67d3c22aa/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:04.753711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724fb7-bc33-4f92-b689-3cff7880ee7e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.754245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724fb7-bc33-4f92-b689-3cff7880ee7e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.754245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724fb7-bc33-4f92-b689-3cff7880ee7e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.755610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 581/1154] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:04.761536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:04.761924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:04.762070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:04.780219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:04.781377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:04.845657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 574/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:05.112000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b724fb7-bc33-4f92-b689-3cff7880ee7e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.209682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b724fb7-bc33-4f92-b689-3cff7880ee7e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.241051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.241658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.241931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.242887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.250033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:05.253717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:14:05.259216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:05.260366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:14:05.261597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 582/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:05.266293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.293363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:05.362935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b89e-ec19-4ddb-835b-946adcf37961 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.396892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.397492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.397769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.398040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.399211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:05.399517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:05.400900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:05.408740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 575/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:05.415286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.511110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daddf03-726d-450a-9e8c-a00878d2c6ac None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.543796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.544401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.544676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.544946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] PATH_INFO: `/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.546038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] RBAC: Authorizing `identity:get_user(user_id=385ffb126f2d4aad9b1e96b67d3c22aa)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:05.547460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:05.549025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 583/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/users/385ffb126f2d4aad9b1e96b67d3c22aa => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:05.555429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.646862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:05.661653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979c325-3888-40a3-8944-bfd3e676c43d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.694265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.694863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.695158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.695431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.759847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:05.761294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:05.846420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:05.858205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 576/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:14:05.864262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:05.929469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:05.963682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e906eb-7a7e-4aa8-9949-b86cf5dac2d5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:05.996302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:05.996919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:05.997200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:05.997474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] PATH_INFO: `/v3/projects/08d6081692b1475f8d37f4f0413b3f34` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:05.998576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] RBAC: Authorizing `identity:get_project(project_id=08d6081692b1475f8d37f4f0413b3f34)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:06.000035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df7b45-ed8e-44d9-80c5-afc915e12042 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:06.001629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 584/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:14:05 2022] GET /identity/v3/projects/08d6081692b1475f8d37f4f0413b3f34 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:06.008339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:06.101558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df9dab-d30c-47f8-a0dd-c7fa9a1cf403 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:06.134046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:06.134631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:06.134906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:06.135173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:06.135913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=08d6081692b1475f8d37f4f0413b3f34, user.id=385ffb126f2d4aad9b1e96b67d3c22aa) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:06.136216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:06.137616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dab389-0ffa-4dba-948f-95ec43f658a0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:06.143100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 577/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:14:06 2022] GET /identity/v3/role_assignments?user.id=385ffb126f2d4aad9b1e96b67d3c22aa&scope.project.id=08d6081692b1475f8d37f4f0413b3f34&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:08.295945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841661cf-e388-43ca-899a-7657c86f8162 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:08.296498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841661cf-e388-43ca-899a-7657c86f8162 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:08.296498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841661cf-e388-43ca-899a-7657c86f8162 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:08.297799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 585/1162] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:08.303067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070542-5067-4595-a45e-9706fb873626 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:08.303455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070542-5067-4595-a45e-9706fb873626 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:08.303655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74070542-5067-4595-a45e-9706fb873626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:08.331366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-74070542-5067-4595-a45e-9706fb873626 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:08.332594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74070542-5067-4595-a45e-9706fb873626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:08.380587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:08.397569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 578/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:08.669554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841661cf-e388-43ca-899a-7657c86f8162 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:08.767238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841661cf-e388-43ca-899a-7657c86f8162 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:08.799869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:08.800458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:08.800734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:08.801125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:08.804944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:08.866416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:08.869634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:08.964144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 586/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:14:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:14:08.970225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74070542-5067-4595-a45e-9706fb873626 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:09.067765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74070542-5067-4595-a45e-9706fb873626 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:09.100651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:09.101398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:09.101893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:09.102217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:09.108609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:09.110017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-22119fff-4b53-444d-969d-0985cbe00da2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:09.121012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:09.122224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:14:09.123276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 579/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:14:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:09.128144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:09.162717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:09.225308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78026d-5b92-4f72-9486-d958b22829f1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:09.257106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:09.258217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:09.258491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:09.258762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:09.259387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:09.259712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:09.261066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d01520-8f0d-4ba1-b502-ac70b4758c30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:09.267916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 587/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 21:14:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:11.391893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdc2d-6c97-45be-b66c-642b94f14855 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.392436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdc2d-6c97-45be-b66c-642b94f14855 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.392436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdc2d-6c97-45be-b66c-642b94f14855 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.393783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 580/1167] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:11.399278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef48d5b-6fbd-41ef-b92d-5e8399baf048 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.399719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef48d5b-6fbd-41ef-b92d-5e8399baf048 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.399980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef48d5b-6fbd-41ef-b92d-5e8399baf048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.424877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bef48d5b-6fbd-41ef-b92d-5e8399baf048 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:11.426072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bef48d5b-6fbd-41ef-b92d-5e8399baf048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:11.490119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 588/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:11.766297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cdc2d-6c97-45be-b66c-642b94f14855 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:11.812668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:11.867709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cdc2d-6c97-45be-b66c-642b94f14855 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:11.900077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:11.900668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:11.900948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:11.901221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:11.901890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:11.903260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c477079-48d9-4cc5-bfb1-257092d56ce3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:11.922108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 581/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Fri Sep 9 21:14:11 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:14.074487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad61ae-6c83-4ea4-b2c6-575571a96f50 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:14.074976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad61ae-6c83-4ea4-b2c6-575571a96f50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:14.075070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad61ae-6c83-4ea4-b2c6-575571a96f50 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:14.076402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 589/1170] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:14.081504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:14.081894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:14.082057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:14.105212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:14.106510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:14.173670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 582/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:14.453820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad61ae-6c83-4ea4-b2c6-575571a96f50 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:14.484162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:14.554085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad61ae-6c83-4ea4-b2c6-575571a96f50 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:14.587265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:14.587906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:14.588186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:14.588460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] PATH_INFO: `/v3/domains/0573ad51a3214ba1a91ef445483d65b5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:14.598482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=0573ad51a3214ba1a91ef445483d65b5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:14.599944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993c1d1-f759-483e-8f5f-1d71cec1dc9b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:14.601606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 590/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Fri Sep 9 21:14:14 2022] GET /identity/v3/domains/0573ad51a3214ba1a91ef445483d65b5 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:14.607343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:14.641476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:14.709000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90d6e-5ec3-436d-b352-a085f06aa668 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:14.742120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:14.742912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:14.743194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:14.743464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:14.744153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:14.745784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c62c7c-b063-4ddb-a558-7d237e70a3b7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:14.773571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 583/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:14:14 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:16.945100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1af73f-66c5-4854-8635-fa055fac46f3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:16.945644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1af73f-66c5-4854-8635-fa055fac46f3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:16.945644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1af73f-66c5-4854-8635-fa055fac46f3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:16.946969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 591/1174] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:16.952426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:16.952838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:16.952973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:16.971312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:16.972519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:17.035980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 584/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:17.311558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1af73f-66c5-4854-8635-fa055fac46f3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:17.402341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:17.412666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1af73f-66c5-4854-8635-fa055fac46f3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:17.445359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:17.445984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.446258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.446528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] PATH_INFO: `/v3/domains/0573ad51a3214ba1a91ef445483d65b5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.447649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] RBAC: Authorizing `identity:get_domain(domain_id=0573ad51a3214ba1a91ef445483d65b5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:17.449063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4686473-ff8b-4cee-ab8c-8b4c4e42782e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:17.450600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 592/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Fri Sep 9 21:14:17 2022] GET /identity/v3/domains/0573ad51a3214ba1a91ef445483d65b5 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:17.456290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:17.473755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:17.554284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b2bcc-b6b5-4442-893b-633e0ae49baa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:17.587437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:17.588002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:17.588285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:17.588557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:17.589215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:17.590634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc61c8c-5f40-474b-80ed-91db73cf7551 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:17.610940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 585/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:14:17 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:14:19.809352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bfecf-c45a-4d00-b5c3-5e366e59b8b5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:19.809871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bfecf-c45a-4d00-b5c3-5e366e59b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:19.809958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bfecf-c45a-4d00-b5c3-5e366e59b8b5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:19.811230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 593/1178] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:19.816472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:19.816895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:19.816997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:19.835994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:19.837182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:19.900824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 586/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:20.170925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bfecf-c45a-4d00-b5c3-5e366e59b8b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.271232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bfecf-c45a-4d00-b5c3-5e366e59b8b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.305013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.305622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.305899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.306171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.316450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:14:20.321398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:20.322520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:14:20.323600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 594/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:20.328623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.330886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.425513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f57d-6b76-450c-a462-587a5b980073 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.458636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.459232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.459555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.459982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.460628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:20.460952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.462315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.469983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 587/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:20.476556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.495805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.575785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d71998-f109-4ab8-95f1-34fc73ec7e7f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.608654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.609270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.609551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.609825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] PATH_INFO: `/v3/users/e9f7de3c689f44c09436338b48d88882` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.623304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] RBAC: Authorizing `identity:get_user(user_id=e9f7de3c689f44c09436338b48d88882)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.624803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.626403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 595/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/users/e9f7de3c689f44c09436338b48d88882 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:20.632518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:20.693166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.730353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cc699-e4bf-4076-a9c9-05dab7b8c42f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:20.762446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:20.763049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:20.763327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:20.763624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:20.827940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:20.829357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:20.887009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:20.923261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 588/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:14:20.929714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.003818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:21.027321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3943f-2c11-4f01-9df9-704948d8000b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.061209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.061830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.062107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.062378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] PATH_INFO: `/v3/projects/59a9042f54a2420bba982710db3a095c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.063478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] RBAC: Authorizing `identity:get_project(project_id=59a9042f54a2420bba982710db3a095c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.064931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17fc4f-ddaf-464b-ad95-9b139097f7ae None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.066545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 596/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:14:20 2022] GET /identity/v3/projects/59a9042f54a2420bba982710db3a095c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:21.072637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:21.167719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093d749-cebd-47ed-a5dd-bd8fcd945b8a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:21.200214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:21.200801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:21.201079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:21.201348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:21.202065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=59a9042f54a2420bba982710db3a095c, user.id=e9f7de3c689f44c09436338b48d88882) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:21.202369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:21.203799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d15828-3d55-4674-89ef-92f4a8e52dfd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:21.208933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 589/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:14:21 2022] GET /identity/v3/role_assignments?user.id=e9f7de3c689f44c09436338b48d88882&scope.project.id=59a9042f54a2420bba982710db3a095c&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:23.337366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d18269-78a1-4c53-8cf5-591aac585d25 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.337962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d18269-78a1-4c53-8cf5-591aac585d25 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.337962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d18269-78a1-4c53-8cf5-591aac585d25 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.339225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 597/1186] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:23.344340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.344757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.344898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.362849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:23.364069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:23.386354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:23.428828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 590/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:23.699028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d18269-78a1-4c53-8cf5-591aac585d25 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:23.800091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d18269-78a1-4c53-8cf5-591aac585d25 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:23.833405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:23.834017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.834290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.834585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.844284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:14:23.849017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:23.850177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:14:23.851446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 598/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:23.856453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:23.952511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381fa4-3b2a-4f43-911c-f68121d6d77a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:23.985370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:23.985963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:23.986242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:23.986513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:23.987169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:23.987468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:23.988844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:23.996942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 591/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:24.004426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.008940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:24.103928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3739-c038-490a-a1ce-8da2aabf094d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.138437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.139059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.139341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.139645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] PATH_INFO: `/v3/users/e9f7de3c689f44c09436338b48d88882` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.140753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e9f7de3c689f44c09436338b48d88882)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.142182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.143771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 599/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:14:24 2022] GET /identity/v3/users/e9f7de3c689f44c09436338b48d88882 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:24.150513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.172798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:24.247928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254b463-2597-483a-9d25-47eef692c03c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.280793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.281404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.281688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.281958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] PATH_INFO: `/v3/projects/59a9042f54a2420bba982710db3a095c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.283065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] RBAC: Authorizing `identity:get_project(project_id=59a9042f54a2420bba982710db3a095c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.284503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f74e2-ec18-4e1c-9e88-5e14a8c3972d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.286109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 592/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:14:24 2022] GET /identity/v3/projects/59a9042f54a2420bba982710db3a095c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:24.292194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:24.359215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:24.388820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbb0f9-4281-4dd9-9be0-1fc77138e4b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:24.421377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:24.422017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:24.422297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:24.422570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] PATH_INFO: `/v3/projects/59a9042f54a2420bba982710db3a095c/users/e9f7de3c689f44c09436338b48d88882/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:24.424460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] RBAC: Authorizing `identity:create_grant(project_id=59a9042f54a2420bba982710db3a095c, user_id=e9f7de3c689f44c09436338b48d88882, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:24.425887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f297a3-db34-49ec-82d8-ecda5b8fb420 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:24.438287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 600/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:14:24 2022] PUT /identity/v3/projects/59a9042f54a2420bba982710db3a095c/users/e9f7de3c689f44c09436338b48d88882/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:14:26.751982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bfe7c-5c02-42ab-9c32-f298a7af7575 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:26.752496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bfe7c-5c02-42ab-9c32-f298a7af7575 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:26.752603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bfe7c-5c02-42ab-9c32-f298a7af7575 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:26.753869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 593/1193] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:14:26.759558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:26.759968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:26.760131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:26.778756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:14:26.779984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:14:26.844501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 601/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:14:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:14:27.118785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bfe7c-5c02-42ab-9c32-f298a7af7575 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:27.207575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:27.220949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bfe7c-5c02-42ab-9c32-f298a7af7575 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:27.253361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:27.253987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:27.254266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:27.254537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:27.265087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:14:27.269871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:14:27.270925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:14:27.272612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 594/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:14:27.277978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:27.336936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:27.375387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1194ed-8268-4aa9-ad58-2c7bdab3d743 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:27.408870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:27.409469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:27.409757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:27.410026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:27.410689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:27.410993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:27.412443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:27.420420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 602/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:27.426253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:27.522382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d79f2-0949-4dc1-9f2d-f3f8b4a737a5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:27.557816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:27.558422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:27.558697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:27.558964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] PATH_INFO: `/v3/users/e9f7de3c689f44c09436338b48d88882` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:27.560090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] RBAC: Authorizing `identity:get_user(user_id=e9f7de3c689f44c09436338b48d88882)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:27.562210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:27.563795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 595/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 21:14:27 2022] GET /identity/v3/users/e9f7de3c689f44c09436338b48d88882 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:27.570536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:27.670292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865973c-5d93-4326-a198-226017ea3653 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:27.692343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:27.703735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:27.704334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:27.704612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:27.704905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:27.774267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:27.775895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:27.872546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 603/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:14:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:14:27.878481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:27.896329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:27.977054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aad2da-6273-4cd1-a897-19674e7252b3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:28.010267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:28.010878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:28.011155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:28.011426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] PATH_INFO: `/v3/projects/59a9042f54a2420bba982710db3a095c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:28.012567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] RBAC: Authorizing `identity:get_project(project_id=59a9042f54a2420bba982710db3a095c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:28.013971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe6b8a-6d93-4348-ab97-1494d8527193 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:28.015589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 596/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 21:14:27 2022] GET /identity/v3/projects/59a9042f54a2420bba982710db3a095c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:28.021567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:14:28.046331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:28.119772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f5e0b-7384-4092-8f0b-e6f430549db7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:14:28.153262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:14:28.153796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:14:28.154074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:14:28.154346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:14:28.155063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=59a9042f54a2420bba982710db3a095c, user.id=e9f7de3c689f44c09436338b48d88882) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:14:28.155369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:14:28.156825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e069b2-0bee-471a-8a5d-8b3376871939 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:14:28.163186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 604/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:14:28 2022] GET /identity/v3/role_assignments?user.id=e9f7de3c689f44c09436338b48d88882&scope.project.id=59a9042f54a2420bba982710db3a095c&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:14:38.005992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=58 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:14:38.159882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=56 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:24.253752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec5dc2-20e7-4888-ba45-6a58baba849d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:24.254382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec5dc2-20e7-4888-ba45-6a58baba849d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:24.254382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec5dc2-20e7-4888-ba45-6a58baba849d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:24.256131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 597/1201] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:24.262126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:24.262618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:24.262618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:24.283373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:24.285652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:24.378204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 605/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:24.679134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec5dc2-20e7-4888-ba45-6a58baba849d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:24.802795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec5dc2-20e7-4888-ba45-6a58baba849d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:24.843441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:24.844133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:24.844395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:24.844664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:24.858033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:24.859920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:24.882476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:24.884397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:24.885688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:24.885688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:24.885688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:24.887946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 598/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:24.891604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:25.068278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0ce2b-ac34-4b2e-b98d-42a4d143cb02 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:25.112358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:25.120488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:25.121115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:25.121397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:25.121712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:25.122376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:25.122684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:25.126770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9a3a-243e-441c-b392-ad27f7cc35ab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:25.145987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 606/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:25.154642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:25.257583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:25.335810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f35d86-4c67-4991-acba-d3a6e8e82cf5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:25.385803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:25.386884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:25.387173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:25.387449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:25.388210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:25.390008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e5c7c-9f57-4cf9-9ef9-0371937ccc3a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:25.418752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 599/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 21:15:25 2022] POST /identity/v3/users => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:15:27.888469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff6262-0f92-4591-8921-d026aea8acff None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.889019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff6262-0f92-4591-8921-d026aea8acff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.889019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff6262-0f92-4591-8921-d026aea8acff None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.890378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 607/1206] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:27.896821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:27.897301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:27.897427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:27.916420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:27.917644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:28.015719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 600/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:28.293273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff6262-0f92-4591-8921-d026aea8acff None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.410423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff6262-0f92-4591-8921-d026aea8acff None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.452586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.453215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.453555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.453818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.466413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:28.471650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:28.472689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:28.473586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 608/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:28.479040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.525186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:28.598847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262d53f-d257-49fc-ac43-5b3d2d3096bb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.648160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.648787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.649069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.649328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.649989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:28.650287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.652059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.661628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 601/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:15:28.667175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.733875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:28.795821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab0ae6-f3e0-422e-8eae-b77740fb4aad None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:28.843138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:28.843719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:28.843991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:28.844282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:28.855835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:28.857214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:28.874681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:28.876151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:28.877171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:28.877171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:28.877171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:28.877171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 609/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:28.882714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:28.997869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:29.000968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da4cff-4c9f-4ef6-91a2-5e42abad6031 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:29.040346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:29.040972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:29.041249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:29.041521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:29.042176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:29.042632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:29.043884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:29.063122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 602/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:29.064052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:29.177300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06622-9e89-42fc-ab6c-1d1741a244ef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:29.178948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:29.218444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:29.219090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:29.219370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:29.219670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:29.228374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:29.229791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:29.243059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:29.244415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:29.245618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 610/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:15:29.250592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:29.365069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aaf54-b726-4c9a-9c67-44c0b8776125 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:29.401908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:29.402559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:29.402856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:29.403127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:29.403914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:29.404300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:29.405905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:29.429750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 603/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:29.431270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:29.536348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8083-7d18-4d74-8a68-db3099145b38 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:29.571971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:29.572541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:29.572823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:29.573099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:29.604770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:29.656472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:29.658122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:29.786231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 611/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:15:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 21:15:29.792438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:29.813018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:29.914946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d525b0-efb8-4c4d-8b86-7e9217e390c8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:29.959895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:29.960750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:29.961051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:29.961347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:29.980632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:29.982484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:30.007007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:30.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:30.009685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:30.009685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:30.009685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:30.009685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 604/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:30.017179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:30.094662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:30.143628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85582dc-40d3-4bb4-83a0-fa6135a22276 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:30.195940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:30.196764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:30.197184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:30.197591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:30.198372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:30.198750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:30.200276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:30.218990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 612/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:30.226042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:30.321891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:30.362167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6e2a1-554c-47f1-9705-845291af20f5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:30.399578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:30.400206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:30.400481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:30.400749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:30.407612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:30.409589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:30.421302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:30.422412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:30.423490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 605/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:30.428943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:30.534497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9b28-43c7-45fe-9b32-c4b9fbe7b2b9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:30.559557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:30.572573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:30.573210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:30.573488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:30.573756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:30.574440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:30.574739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:30.576192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b8f59-1b21-48a0-bfa5-965ea88f22e8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:30.589212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 613/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:30.595270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:30.709245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676c65-120f-4340-ba8c-1a35579eb7eb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:30.742611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:30.749975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:30.750693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:30.751007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:30.751347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:30.753038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8ac80d1c221e483989f1e4f5d41a0897) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:30.753393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:30.755420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40836a-79c6-4f0e-85b1-c01a61ac232e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:30.761918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 606/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:15:30 2022] GET /identity/v3/role_assignments?user.id=8ac80d1c221e483989f1e4f5d41a0897&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:32.937965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c4de-7cad-4ae1-8571-202f444676dc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:32.938586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c4de-7cad-4ae1-8571-202f444676dc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:32.938586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c4de-7cad-4ae1-8571-202f444676dc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:32.940569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 614/1220] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:32.945878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:32.946464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:32.946567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:32.968010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:32.969454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:33.048766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 607/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:33.351338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c4de-7cad-4ae1-8571-202f444676dc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:33.464809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c4de-7cad-4ae1-8571-202f444676dc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:33.500012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:33.500649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:33.500926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:33.501196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:33.512069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:33.517049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:33.518804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:33.520233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:33.521491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 615/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:15:33.526898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:33.644405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f90639-0d92-47f7-8323-022c9e90eb22 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:33.670359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:33.683832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:33.684462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:33.684734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:33.685034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:33.685757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:33.686114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:33.687558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:33.696928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 608/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:33.703016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:33.759229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:33.818456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76847b-0823-4bb7-a51a-e38d25b1fba4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:33.857243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:33.858057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:33.858446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:33.858840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:33.870646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:33.872214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:33.887662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:33.888825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:33.890258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:33.890258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:33.890258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:33.890258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 616/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:33.895335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:33.998785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609a4e-facf-4223-bb1f-47c7ee8cd7bb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.038994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.039894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.040170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.040440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.041102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:34.041400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.042818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.060162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 609/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:34.061718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.174873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:34.183015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5083b-f677-4c5e-99cb-dc25b421d26e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.221618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.222254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.222584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.222856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.231126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.233038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:34.245599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:34.251572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:34.253376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 617/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:34.254110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.307336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:34.369567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345dd0a-2e6d-46de-afcd-518161cf24b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.405163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.405818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.406097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.406412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.407137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:34.407455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.408980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.427688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 610/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:15:34.433604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.541246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401b78a-d6ef-43c5-8c28-2cf600f0da51 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.575758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.576357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.576640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.576910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.588806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.590962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:34.613115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:34.619119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:34.620464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:34.620464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:34.620464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:34.620464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 618/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:15:34.620989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.734343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2f3b2-894e-4ae7-bd30-c4bba30d24ca None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.737575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:34.773091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.773724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.774009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.774289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.774919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:34.775224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.776805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.789683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 611/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:34.795419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:34.806134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:34.911555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bae24-afbd-41c8-a3a6-8a5c9092e14f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:34.950857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:34.951475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:34.951786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:34.952076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:34.966987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:34.969654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:34.984828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:34.987113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:34.989221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 619/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:34.993511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:35.099669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68579d-9e79-4f0b-b42f-92bfd33769d3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:35.134249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:35.134932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:35.135297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:35.135598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:35.136303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:35.136681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:35.138276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c2b53-d03b-4417-9be8-94605cb54e3e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:35.156136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 612/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:35.157373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:35.231595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:35.273387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5ee1-48b1-41aa-99ff-ec6416bd2c1c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:35.312814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:35.313639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:35.313940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:35.314399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8ac80d1c221e483989f1e4f5d41a0897/roles/86248f9a95e44d1a9730d95b7f9b40e9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:35.330613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=8ac80d1c221e483989f1e4f5d41a0897, role_id=86248f9a95e44d1a9730d95b7f9b40e9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:35.332173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f54f27-7e27-4a89-b06c-a559dbaf4e4c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:35.346079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 620/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:15:35 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8ac80d1c221e483989f1e4f5d41a0897/roles/86248f9a95e44d1a9730d95b7f9b40e9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:37.644061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c75-7f46-4327-8125-48dc3c8a3dda None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:37.644507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c75-7f46-4327-8125-48dc3c8a3dda None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:37.644608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c75-7f46-4327-8125-48dc3c8a3dda None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:37.646404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 613/1233] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:37.651947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:37.652399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:37.652501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:37.672604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:37.674640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:37.752128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 621/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:38.028284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0c75-7f46-4327-8125-48dc3c8a3dda None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:38.059376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:38.143979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0c75-7f46-4327-8125-48dc3c8a3dda None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:38.187402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:38.188135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:38.188414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:38.188679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:38.200469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:38.205375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:38.206441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 21:15:38.207306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:38.212702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 614/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:38.212849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:38.264654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:38.327541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657d02-3309-4634-b8fa-123ccfa26bf3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:38.368064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:38.368671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:38.368948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:38.369214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:38.369892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:38.370179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:38.371613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:38.382277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 622/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:15:38.388359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:38.482656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:38.501400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46a62-7ec2-45d9-98c6-6a11e319a447 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:38.536916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:38.537548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:38.537831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:38.538102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:38.549776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:38.551225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:38.566449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:38.572302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:38.573440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:38.573440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:38.573440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:38.573440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 615/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:38.573733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:38.684815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ace6-0472-4a04-be44-bb5ec8fb35d1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:38.693846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:38.722551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:38.723164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:38.723444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:38.723760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:38.725218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:38.725218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:38.726067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:38.738703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 623/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:38.744518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:38.864258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb19384-6183-42e1-b0e7-5be5c5ca0342 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:38.902102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:38.903032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:38.903504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:38.903983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:38.907330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:38.914082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:38.916487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:38.930120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:38.931349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:38.932635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:38.934572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 616/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:38.941864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:39.074622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03ce84-5e75-4483-b9ce-2585d5b822d8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:39.115588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:39.116027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:39.116310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:39.116583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:39.117317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:39.117622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:39.119065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:39.138021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 624/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:39.144205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:39.250949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c848-d95c-4974-acbb-a1cb6efa1f56 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:39.293187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:39.293845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:39.294164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:39.294439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:39.364636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:39.372285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:39.373767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:39.479098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 617/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:15:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:15:39.486258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:39.515590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:39.599872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f271fb-8b6e-4be1-b424-8fc7b7188efa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:39.635941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:39.636572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:39.636861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:39.637128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:39.648298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:39.649755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:39.665320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:39.666560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:39.667462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:39.667462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:39.667462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:39.667462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 625/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 21:15:39.672453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:39.776126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:39.784341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa300b51-ddd5-4c3f-9e41-6f3a009ae0fd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:39.820527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:39.821131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:39.821407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:39.821671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:39.822310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:39.822604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:39.823994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:39.835683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 618/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:15:39.841586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:39.931490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:39.955820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385d0a-b758-44a1-9e0c-22bc8db9a47d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:40.000998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:40.002144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:40.002465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:40.002775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:40.011180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:40.012926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:40.029033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:40.030022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:40.031037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 626/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:40.036959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:40.155314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adf4f9-d91c-4be8-a0ff-f5e94b46f586 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:40.195102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:40.195771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:40.196061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:40.196356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:40.197091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:40.197474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:40.198997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97130868-58cb-451f-bb42-5caee41cc555 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:40.212284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 619/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:15:40.218375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:40.332088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afdbc3-08e2-490c-a799-301a71946d85 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:40.374061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:40.374713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:40.375018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:40.375301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:40.376086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] RBAC: Adding query filter params (role.id=86248f9a95e44d1a9730d95b7f9b40e9, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8ac80d1c221e483989f1e4f5d41a0897) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:40.376484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:40.378109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0176e7-1f09-4668-a9f7-6471b4d0fe4f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:40.385040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 627/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:15:40 2022] GET /identity/v3/role_assignments?user.id=8ac80d1c221e483989f1e4f5d41a0897&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=86248f9a95e44d1a9730d95b7f9b40e9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:42.753439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c0d3-2d8f-43fa-baf3-5fc9756ebf03 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:42.753984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c0d3-2d8f-43fa-baf3-5fc9756ebf03 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:42.753984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c0d3-2d8f-43fa-baf3-5fc9756ebf03 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:42.755308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 620/1247] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:15:42.761623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:42.762097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:42.762358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:42.769018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:42.782904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:42.784132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:42.853228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 628/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:43.143167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c0d3-2d8f-43fa-baf3-5fc9756ebf03 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:43.162860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:43.265441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c0d3-2d8f-43fa-baf3-5fc9756ebf03 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:43.304131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:43.304756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:43.305034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:43.305295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:43.317180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:43.321965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:43.323105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:43.324255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 621/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:43.329819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:43.448954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5575c8c-0c96-4499-8b05-14fb9b58b319 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:43.481951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:43.488050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:43.488697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:43.488938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:43.489209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:43.489872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:43.490173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:43.491549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:43.500062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 629/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:43.507014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:43.588155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:43.615649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761a2da-63e7-4c50-a090-95ebd405077f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:43.652068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:43.652762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:43.652977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:43.653245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:43.664194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:43.665766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:43.681713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:43.682780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:43.684054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:43.684054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:43.684054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:43.684054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 622/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:43.689324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:43.809592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1f0e-0df8-47c5-bee3-342edaac43e3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:43.848960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:43.849638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:43.849982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:43.850319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:43.851025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:43.851361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:43.852913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:43.867551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 630/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:43.873192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:43.988977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6528c-743b-42af-81da-d176bcf7a645 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:44.023679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:44.033409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:44.034289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:44.034573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:44.034875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:44.043941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:44.045422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:44.058294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:44.064690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:44.066843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 623/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:44.067605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:44.126187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:44.174656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc834f6-e8e6-4c4d-9e23-7e9cdae05641 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:44.212904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:44.213524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:44.213804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:44.214078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:44.214823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:44.215133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:44.216626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:44.237326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 631/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:44.243359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:44.350044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e5123-ab2e-4c61-b6cf-6b41d87c7371 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:44.383491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:44.384116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:44.384391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:44.384659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:44.440082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:44.453165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:44.454627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:44.555938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 624/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:15:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:15:44.561891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:44.664037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2b31-23a7-4220-a004-9958f32a913b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:44.676743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:44.698319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:44.698939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:44.699217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:44.699487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:44.710128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:44.711600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:44.726439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:44.727578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:44.728693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:44.728693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:44.728693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:44.728693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 632/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:44.733272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:44.817180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:44.828845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4af9-91e4-44d9-92cb-863d556bed01 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:44.862835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:44.863412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:44.863723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:44.863992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:44.864623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:44.864922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:44.866266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:44.877013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 625/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:44.882735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:44.983152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eff3d8-3f00-41b5-a2eb-d25e95fac761 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:45.022962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:45.023605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:45.023885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:45.024154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:45.030604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:45.032070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:45.043188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:45.044279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:45.045552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 633/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:45.050359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:45.147208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a50a0-87b5-4eec-a5f2-e9117e8731b4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:45.179002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:45.179612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:45.179887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:45.180153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:45.180831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:45.181130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:45.182537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fedd69-4f44-4fbd-918d-8be918753bac None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:45.194952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 626/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:45.200932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:45.226952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:45.304314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36beaaa-be7e-4b35-9405-e5d767ed502d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:45.337432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:45.338023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:45.338301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:45.338573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:45.339292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8ac80d1c221e483989f1e4f5d41a0897) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:45.339621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:45.341039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7491f-bc82-41db-afda-97bfd8cd1913 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:45.346656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 634/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:15:45 2022] GET /identity/v3/role_assignments?user.id=8ac80d1c221e483989f1e4f5d41a0897&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:47.460806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453638-7e44-4c25-af1d-0a605dedd270 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:47.461356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453638-7e44-4c25-af1d-0a605dedd270 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:47.461356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453638-7e44-4c25-af1d-0a605dedd270 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:47.462657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 627/1261] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:47.467864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:47.468260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:47.468416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:47.487325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:47.488552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:47.553149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 635/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:47.819362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453638-7e44-4c25-af1d-0a605dedd270 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:47.830663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:47.918277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453638-7e44-4c25-af1d-0a605dedd270 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:47.949957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:47.950563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:47.950836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:47.951101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:47.961186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:47.966366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:47.967650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:47.968659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 628/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:47.975095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:47.999001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:48.090308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c117d-bf34-4f19-b320-518dcbd31635 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.123841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.124439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.124714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.124982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.125637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:48.126562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.127950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.135635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 636/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:48.141571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:48.207403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:48.240406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af856d-afc6-4ba6-aa51-f99dec2c25c4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.272858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.273465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.273739 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.274006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.284235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.285648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:48.300326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:48.301420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:48.302534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:48.302534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:48.302534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:48.302534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 629/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:48.307199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:48.384842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:48.406101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de00cd9-01c2-4182-b1d8-9e39e2bf5e6c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.438715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.439309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.439604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.439876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.440504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:48.440801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.442153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.453327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 637/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:48.459658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:48.554861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942eb98d-4bf5-4f0b-87be-415ad92aefa9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.573317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:48.587216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.587851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.588130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.588395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.596071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.597472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:48.608152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:48.609221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:48.610485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 630/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:48.615214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:48.714770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dc879-eb2e-41d8-a356-aad0a2a99c2d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.747813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.748410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.748691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.748963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.749681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:48.749988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.751977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.760368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:48.769379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 638/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:48.775392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:48.871318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa881b-1ca7-47bf-84fc-600036580659 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:48.904089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:48.904675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:48.904953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:48.905227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:48.915499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:48.916950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:48.930716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:48.931862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:48.933092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:48.933092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:48.933092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:48.933092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 631/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:48.937420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:49.047855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029eaf8-1539-484a-8af2-da339957f529 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:49.081770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:49.082372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:49.082656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:49.082930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:49.083591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:49.083900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:49.085399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:49.096366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 639/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:49.102033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:49.133320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:49.198284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76032-69b6-43a7-9e61-011e452fecf3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:49.229956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:49.230578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:49.230843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:49.231107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:49.238066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:49.239467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:49.250220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:49.251880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:49.253206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 632/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:49.257815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:49.315054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:49.355590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19c17-3ada-487d-97a3-b02a97945905 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:49.388538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:49.389128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:49.389411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:49.389683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:49.390367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:49.390672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:49.392436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f261d-9c4a-439f-a641-c4ce8c675240 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:49.404182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 640/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:49.410277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:49.498666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:49.508258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6de15-ad6d-473c-b1fa-8c46e122e751 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:49.541126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:49.541766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:49.542046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:49.542319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] PATH_INFO: `/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8ac80d1c221e483989f1e4f5d41a0897/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:49.544180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] RBAC: Authorizing `identity:create_grant(project_id=f864ecf3c22841c98fde56ead0bbe826, user_id=8ac80d1c221e483989f1e4f5d41a0897, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:49.545597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54135b4d-7f0e-462e-a24b-4ef527ac536a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:49.558482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 633/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 21:15:49 2022] PUT /identity/v3/projects/f864ecf3c22841c98fde56ead0bbe826/users/8ac80d1c221e483989f1e4f5d41a0897/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:15:51.692289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca1d6-81a1-43be-a330-4ad5ccc83d77 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:51.692781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca1d6-81a1-43be-a330-4ad5ccc83d77 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:51.692876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca1d6-81a1-43be-a330-4ad5ccc83d77 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:51.694171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 641/1274] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:51.699165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:51.699627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:51.699761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:51.717675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:51.718856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:51.784047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 634/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:52.063270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca1d6-81a1-43be-a330-4ad5ccc83d77 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.173883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca1d6-81a1-43be-a330-4ad5ccc83d77 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.190436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:52.207427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.208073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.208353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.208624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.218317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 21:15:52.223689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:52.224753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:52.225935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 642/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:52.230721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.313947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:52.333247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522ef0-8bac-4f3d-89d4-7ef0f12a1e65 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.366239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.366836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.367114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.367383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.368068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:52.368366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:52.369712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:52.377661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 635/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:52.383454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.483196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c89035-3ca1-4399-936d-364c5d7c5331 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.517109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.517724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.518000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.518268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.528085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:52.529516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:52.544415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:52.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:52.546551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:52.546551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:52.546551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:52.546551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 643/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:52.551181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.648132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1175-7284-4125-94d6-20e9ca8b7f3d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.679835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.680426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.680699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.680965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.681594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:52.681890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:52.683238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:52.695063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 636/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:52.700675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.730808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:52.798158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb24b3-6319-4c45-b9f7-1e8618567174 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.830959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.831583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.831862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.832129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.839398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:52.840876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 21:15:52.852040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:52.853151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:52.854486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 644/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:52.858804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:52.863312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:52.955521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0ed50-5e18-4781-a927-86525badc643 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:52.988263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:52.988857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:52.989136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:52.989409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:52.990131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:52.990436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:52.991888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.008912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 637/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 21:15:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:53.014533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:53.105369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:53.115752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad26605-387e-4819-bdfe-231e62c8c573 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:53.149032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:53.149627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:53.149909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:53.150186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:53.215813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:53.217188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.304120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:53.314406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 645/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 21:15:53.320193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:53.379772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:53.417010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf261b-e22a-4f8a-98ee-fcee7b6c5e49 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:53.449752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:53.450362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:53.450640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:53.450908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:53.460908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:53.462321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 21:15:53.477533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:53.478568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 21:15:53.479607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 21:15:53.479607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 21:15:53.479607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:53.479607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 638/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:53.484320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:53.580872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c864a-7663-43c3-8959-480e8f065d90 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:53.614458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:53.615046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:53.615322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:53.615619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:53.616249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:53.616546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:53.617898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.628906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 646/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:53.634439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:53.731269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689355-f99b-49ea-9b5c-ff0619769b24 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:53.744334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:53.763654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:53.764263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:53.764539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:53.764807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:53.772035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:53.773462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 21:15:53.785031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:53.786067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Sep 09 21:15:53.787144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 639/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:53.791895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:53.841275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:53.890706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a46ff8-5be6-4e11-8210-ead19d820c45 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:53.926313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:53.926908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:53.927186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:53.927456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:53.928168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:53.928470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:53.929889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891cb6c-63b0-4c42-a0d3-3d4e80f7a04d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:53.941736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 647/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:53.947775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:54.051160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab3f33-c1e6-49e7-bc14-994892ebeef6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:54.084449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:54.085035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:54.085314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:54.085582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:54.086299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] RBAC: Adding query filter params (role.id=b1aece7abc4d43b78033c71e40fda24e, scope.project.id=f864ecf3c22841c98fde56ead0bbe826, user.id=8ac80d1c221e483989f1e4f5d41a0897) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:54.086602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:54.088064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb653c8-2efb-474d-b78d-eaf92a01283c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:54.093564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 640/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 21:15:53 2022] GET /identity/v3/role_assignments?user.id=8ac80d1c221e483989f1e4f5d41a0897&scope.project.id=f864ecf3c22841c98fde56ead0bbe826&role.id=b1aece7abc4d43b78033c71e40fda24e => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:56.253955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77cb36-a81d-4d58-9c49-2cf7340b98f6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:56.254486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77cb36-a81d-4d58-9c49-2cf7340b98f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:56.254486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77cb36-a81d-4d58-9c49-2cf7340b98f6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:56.255884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 648/1288] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:56.261033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:56.261475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:56.261568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:56.279744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:56.280926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:56.313057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:56.350056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 641/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:56.618597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77cb36-a81d-4d58-9c49-2cf7340b98f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:56.717709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77cb36-a81d-4d58-9c49-2cf7340b98f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:56.728533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:56.751293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:56.751927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:56.752204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:56.752474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:56.753045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:56.754399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:15:56.762889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:15:56.764117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:15:56.765245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 649/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:15:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:15:56.771060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:56.864807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197524e0-d4a1-4ef6-a0dd-758cb0458b5a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:56.897267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:56.897788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:56.898061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:56.898327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:56.898939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:56.899234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:56.900678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa964c79-f866-40c5-afbe-5a0d26b1309c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:56.905234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 642/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 21:15:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:56.911403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:57.022633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0b17-1c9a-40ce-8087-1d006d076e22 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:57.059443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:57.060099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:57.060379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:57.060654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:57.061280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:15:57.061580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:57.063463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8fa5d-3cef-4888-9291-9ff41c5c872b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:57.067827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 650/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 21:15:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:15:59.213723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888ae4-af26-4123-ba6f-3b7e150ee2d7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:59.214253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888ae4-af26-4123-ba6f-3b7e150ee2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:59.214253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888ae4-af26-4123-ba6f-3b7e150ee2d7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:59.215627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 643/1293] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:15:59.221458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3c0b-869a-4510-bd4c-d22306ce56bb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:59.221834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3c0b-869a-4510-bd4c-d22306ce56bb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:59.222006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b3c0b-869a-4510-bd4c-d22306ce56bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:59.240508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-458b3c0b-869a-4510-bd4c-d22306ce56bb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:15:59.241680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-458b3c0b-869a-4510-bd4c-d22306ce56bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:15:59.270822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:59.309190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 651/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:15:59.577168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e888ae4-af26-4123-ba6f-3b7e150ee2d7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:15:59.607091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:15:59.678399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e888ae4-af26-4123-ba6f-3b7e150ee2d7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:15:59.711117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:15:59.711741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:15:59.712022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:15:59.712291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:15:59.712930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:15:59.714277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07fc7f-9ad2-4d84-bfd7-a6dc9d941403 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:15:59.728691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 644/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 21:15:59 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:01.920023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f47c2-62f8-4c22-97e8-dd54eca3cf82 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:01.920530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f47c2-62f8-4c22-97e8-dd54eca3cf82 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:01.920640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f47c2-62f8-4c22-97e8-dd54eca3cf82 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:01.921898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 652/1296] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:16:01.927319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:01.927780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:01.927886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:01.945488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:01.946653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:02.025996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 645/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:16:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:16:02.298166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f47c2-62f8-4c22-97e8-dd54eca3cf82 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:02.402482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f47c2-62f8-4c22-97e8-dd54eca3cf82 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:02.434554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:02.435169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:02.435447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:02.435814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] PATH_INFO: `/v3/services/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:02.436394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:02.438408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:16:02.446906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:16:02.448071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:02.448961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 653/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:16:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:16:02.454924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:02.473154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:02.552724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5932e-99fb-4dbc-ad16-619076e3aaf3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:02.586111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:02.586707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:02.586990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:02.587266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:02.587925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:16:02.588231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:02.589583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:02.594646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 646/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 21:16:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:16:02.600584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:02.632605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:02.701869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b4d58-e978-4832-a320-1e25a831a2da None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:02.733877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:02.735294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:02.735597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:02.735872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:02.736564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] RBAC: Adding query filter params (interface=public, service_id=9e31893540e544f88de4258b41595e24, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:16:02.736868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:02.738221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9567-1761-4288-bcff-e24ec7018087 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:02.743041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 654/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 21:16:02 2022] GET /identity/v3/endpoints?service_id=9e31893540e544f88de4258b41595e24&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:16:02.748697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:02.827398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:02.845258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7af15-3505-4e8c-8ce9-ff2ada712274 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:02.878542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:02.879125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:02.879397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:02.879689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:02.880280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:02.881613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69727341-1ef0-42ad-926d-c6de0262e478 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:02.888123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 647/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 21:16:02 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:16:05.037053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d7de-bdec-4cf1-9d25-16e19266acd8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:05.037647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d7de-bdec-4cf1-9d25-16e19266acd8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:05.037647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67d7de-bdec-4cf1-9d25-16e19266acd8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:05.038933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 655/1302] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:16:05.044135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:05.044581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:05.044682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:05.062542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:16:05.063730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:16:05.127280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 648/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:16:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 21:16:05.398696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67d7de-bdec-4cf1-9d25-16e19266acd8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:05.490983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:05.497654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67d7de-bdec-4cf1-9d25-16e19266acd8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:05.529822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:05.530440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:05.530718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:05.531481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:05.532094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:05.533462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 21:16:05.542107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 21:16:05.543305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Sep 09 21:16:05.544496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 656/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 21:16:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 21:16:05.549817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:05.622443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:05.646306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7e41a-914c-4e75-95e0-ef28ef963870 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:05.679633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:05.680236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:05.680519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:05.680792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:05.681419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:16:05.681720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:05.683077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186eb05-cbe8-4d2f-a946-84c71a384718 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:05.688062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 649/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 21:16:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:16:05.693861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:16:05.794898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa46a-2ba3-443d-9af7-6ae9cce19649 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:16:05.827133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:16:05.828291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:16:05.828630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:16:05.828898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:16:05.829549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:16:05.830897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb220ee-d3ae-4f13-bc2f-b0efee764eff None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:16:05.850281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 657/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:16:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:16:15.685210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=30 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:16:15.847167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:10.025243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d00-ad31-455b-87aa-e6495ca28b26 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:10.025737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d00-ad31-455b-87aa-e6495ca28b26 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:10.025822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d00-ad31-455b-87aa-e6495ca28b26 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:10.032050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 650/1307] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:10.033337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4329-bf89-4248-b3ac-089cd1b9a298 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:10.033751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4329-bf89-4248-b3ac-089cd1b9a298 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:10.033912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd4329-bf89-4248-b3ac-089cd1b9a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:10.139446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-77cd4329-bf89-4248-b3ac-089cd1b9a298 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:10.140725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-77cd4329-bf89-4248-b3ac-089cd1b9a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:10.249961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 658/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:10.548494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:10.548860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:10.549031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:10.550580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 651/1309] 10.222.0.46 () {54 vars in 965 bytes} [Fri Sep 9 21:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:10.557037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb74da-2de9-466b-95b3-c9d58242f6ad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:10.557331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb74da-2de9-466b-95b3-c9d58242f6ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:10.557844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb74da-2de9-466b-95b3-c9d58242f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:10.659949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3fbb74da-2de9-466b-95b3-c9d58242f6ad None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:10.661180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3fbb74da-2de9-466b-95b3-c9d58242f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:10.685242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:10.751393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 659/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Fri Sep 9 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:10.758580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:10.760336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:10.879627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879d8f7-bd3a-4c7a-aee5-dfaf4a4bd9b3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:10.960956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:10.962150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:10.962549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:10.962822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:10.964945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:11.026766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:11.087062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:11.088613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454ca90-2e7d-4b52-bd74-5659bc0d9374 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:11.108633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 652/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:13.591796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.592340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.592340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.593689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 660/1312] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:13.599047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6dfb-ec78-4def-ab8f-0049de0f9198 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:13.599459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6dfb-ec78-4def-ab8f-0049de0f9198 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:13.599732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6dfb-ec78-4def-ab8f-0049de0f9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:13.681726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9adf6dfb-ec78-4def-ab8f-0049de0f9198 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:13.683213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9adf6dfb-ec78-4def-ab8f-0049de0f9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:13.780936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 653/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:17:14.069459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:14.071050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:14.174530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6e21d-9891-4fc2-8a3b-fdf5eb3f1ee1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:14.248088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:14.250490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:14.251126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:14.251421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:14.251731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:14.253862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:14.362415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:14.363781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201348b-2bdb-4d5c-8356-699e038e1fbf service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:14.381327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 661/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:16.877869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:16.878397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:16.878397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:16.879802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 654/1315] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:16.885244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66118e-4f9d-4575-a9bf-0e851feae302 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:16.886322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66118e-4f9d-4575-a9bf-0e851feae302 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:16.886517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66118e-4f9d-4575-a9bf-0e851feae302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:16.953020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8a66118e-4f9d-4575-a9bf-0e851feae302 None None] MFA Rules not processed for user `41b5e9ce17744df388fc2899d20c97a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:16.954211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8a66118e-4f9d-4575-a9bf-0e851feae302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:17.009621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:17.054917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 662/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:17.338173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:17.339791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:17.349398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:17.450337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee15cb5-ea1a-4635-a57f-bf192ccc9524 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:17.521227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:17.521819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:17.522096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:17.522362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:17.523989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:17.584558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:17.631782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:17.633243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247a04c-3931-4fbe-92dd-a8477463c510 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:17.651090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 655/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:20.334869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:20.335404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:20.335404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:20.336822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 663/1318] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:20.342702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d608-bb5d-4bca-8312-744695a2a6b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:20.343123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d608-bb5d-4bca-8312-744695a2a6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:20.343260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d608-bb5d-4bca-8312-744695a2a6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:20.408165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c0f0d608-bb5d-4bca-8312-744695a2a6b5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:20.409382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0f0d608-bb5d-4bca-8312-744695a2a6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:20.503181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 656/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:20.788638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:20.790193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:20.907025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28c402-a738-4daa-ae89-c29ec498d173 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:20.980786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:20.981392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:20.981672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:20.981944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:20.983599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:20.990525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:21.092234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:21.093659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf07c55-8ee8-4bd9-97ab-19faf15e3374 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:21.112304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 664/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:23.905519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662197e-579a-4627-ae66-f6effc157e9c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:23.906017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662197e-579a-4627-ae66-f6effc157e9c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:23.906119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662197e-579a-4627-ae66-f6effc157e9c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:23.907432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 657/1321] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:23.912782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c46-80df-4d0b-9ebd-e110a712a4e5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:23.913193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c46-80df-4d0b-9ebd-e110a712a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:23.913334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c46-80df-4d0b-9ebd-e110a712a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:23.985191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6e250c46-80df-4d0b-9ebd-e110a712a4e5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:23.986426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e250c46-80df-4d0b-9ebd-e110a712a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:24.048396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:24.092291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 665/1322] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:24.392157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:24.392543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:24.392880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:24.394282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 658/1323] 10.222.0.46 () {54 vars in 965 bytes} [Fri Sep 9 21:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:24.400279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfed2-1c1b-4084-8dc7-4a10c5831312 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:24.401195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfed2-1c1b-4084-8dc7-4a10c5831312 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:24.401480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dfed2-1c1b-4084-8dc7-4a10c5831312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:24.477156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-594dfed2-1c1b-4084-8dc7-4a10c5831312 None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:24.478356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-594dfed2-1c1b-4084-8dc7-4a10c5831312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:24.553748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 666/1324] 10.222.0.46 () {58 vars in 1062 bytes} [Fri Sep 9 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:17:24.563316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:24.564864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:24.584381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:24.669339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34742e8-f4dc-437b-8927-14593d7cffd5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:24.742796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:24.743402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:24.743707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:24.743976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:24.746290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:24.817384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:24.856050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:24.857513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c28740-d0e2-476f-85dc-75e82a8b8428 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:24.875640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 659/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:27.325709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:27.326213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:27.326213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:27.327540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 667/1326] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:27.332794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a83b-2fe8-4a2f-8b6f-6ccf5c8bcbde None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:27.333204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a83b-2fe8-4a2f-8b6f-6ccf5c8bcbde None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:27.333490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a83b-2fe8-4a2f-8b6f-6ccf5c8bcbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:27.405670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1967a83b-2fe8-4a2f-8b6f-6ccf5c8bcbde None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:27.406941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1967a83b-2fe8-4a2f-8b6f-6ccf5c8bcbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:27.522700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 660/1327] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:27.820469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:27.822397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:27.905998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:27.962506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3b7d1-793d-412b-a870-ca581a7a46e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:28.063478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:28.064108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:28.064397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:28.064657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:28.066274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:28.206638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:28.209473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:28.211079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb607a-2719-472d-9670-6512e78a9c5a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:28.233962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 668/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:30.481893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:30.482452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:30.482452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:30.483871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 661/1329] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:30.489596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493d86-f049-41ca-a469-18f8928d1f45 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:30.490071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493d86-f049-41ca-a469-18f8928d1f45 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:30.490194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493d86-f049-41ca-a469-18f8928d1f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:30.561269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0f493d86-f049-41ca-a469-18f8928d1f45 None None] MFA Rules not processed for user `41b5e9ce17744df388fc2899d20c97a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:30.562497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f493d86-f049-41ca-a469-18f8928d1f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:30.665214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 669/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:30.954757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:30.956432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:30.988396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:31.060897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a6ba3-b1d5-48d2-9bc4-8a899e999a34 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:31.140167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:31.140776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:31.141315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:31.141589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:31.143211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:31.224736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:31.257311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:31.259011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ce2bb-17c8-4de2-a684-3039bb03c53d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:31.278121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 662/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:33.840991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:33.841519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:33.841519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:33.842860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 670/1332] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:33.848029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38138a-8355-4447-b5ff-3c4218e4be20 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:33.848410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38138a-8355-4447-b5ff-3c4218e4be20 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:33.848581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38138a-8355-4447-b5ff-3c4218e4be20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:33.913704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6a38138a-8355-4447-b5ff-3c4218e4be20 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:33.914920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6a38138a-8355-4447-b5ff-3c4218e4be20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:34.010566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 663/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:34.293247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:34.294809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:34.363423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:34.398022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e678472-5d81-4546-a5e3-2f3b74574984 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:34.473050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:34.473652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:34.473933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:34.474213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:34.475844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:34.585050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:34.586413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94799e56-b1cc-46db-8ce4-ece60871ebcb service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:34.597133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:34.606185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 671/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:36.809607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:36.810681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:36.810681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:36.811510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 664/1335] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:36.817374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544263-c6e4-45f9-9e1c-16a874498a9c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:36.817830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544263-c6e4-45f9-9e1c-16a874498a9c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:36.817934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17544263-c6e4-45f9-9e1c-16a874498a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:36.888534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-17544263-c6e4-45f9-9e1c-16a874498a9c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:36.889730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17544263-c6e4-45f9-9e1c-16a874498a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:37.004993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 672/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:37.300514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:37.302067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:37.346920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:37.413740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ef65-fc89-4ccc-9a88-737750694381 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:37.486542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:37.487147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:37.487425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:37.487734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:37.489352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:37.576234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:37.603148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:37.604624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeb02d-cc55-4273-a1e7-5c318988c8d0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:37.624062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 665/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:40.264523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.265080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.265080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.266703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 673/1338] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:40.271872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f38a3-f779-4129-b00b-c23ede6fa981 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.272266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f38a3-f779-4129-b00b-c23ede6fa981 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.272418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f38a3-f779-4129-b00b-c23ede6fa981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.336165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-312f38a3-f779-4129-b00b-c23ede6fa981 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:40.338007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-312f38a3-f779-4129-b00b-c23ede6fa981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.433548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 666/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:40.716058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:40.717616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:40.794467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:40.823418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef70b-8fa4-4dd5-9c07-a6427a1bdafb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:40.897940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:40.898540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:40.898816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:40.899082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:40.901213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:41.012428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:41.013773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c40872-94b2-4a1f-83c9-ef40c4d55675 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:41.031879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 674/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:45.126650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:45.127182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:45.127182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:45.128557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 667/1341] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:17:45.134742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cff46-fd3d-4f06-8e38-c19ab29d9081 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:45.135191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cff46-fd3d-4f06-8e38-c19ab29d9081 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:45.135313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cff46-fd3d-4f06-8e38-c19ab29d9081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:45.141104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:45.204194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4b2cff46-fd3d-4f06-8e38-c19ab29d9081 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:45.205358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b2cff46-fd3d-4f06-8e38-c19ab29d9081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:45.304306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 675/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:45.587570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:45.589203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:45.642303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:45.699074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037512e-8d9d-41d1-bd2d-3cbfd08b48bb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:45.773582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:45.774182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:45.774987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:45.775393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:45.777014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:45.871427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:45.887792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:45.889230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55426c6b-6f4d-4ced-babc-0299ac1e1e87 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:45.908369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 668/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:48.851510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:48.852084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:48.852084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:48.854459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 676/1344] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:48.859673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c743f-5850-4d08-b7bc-ce8ca0272f6d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:48.860090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c743f-5850-4d08-b7bc-ce8ca0272f6d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:48.860217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c743f-5850-4d08-b7bc-ce8ca0272f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:48.926220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-088c743f-5850-4d08-b7bc-ce8ca0272f6d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:48.927426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-088c743f-5850-4d08-b7bc-ce8ca0272f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:49.036676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 669/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:49.324491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:49.326048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:49.417382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:49.434794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc59e3-9473-4917-88ad-a9cfc356caa3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:49.511232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:49.511785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:49.512061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:49.512326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:49.514198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:49.630432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:49.632098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7285ff7-ba2d-44c7-99b6-dc18858f5300 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:49.651157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 677/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:54.058457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:54.058957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:54.059074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:54.060416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 670/1347] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:54.065904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd056-d05c-415e-8b08-6b2c6d1e8fbe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:54.066395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd056-d05c-415e-8b08-6b2c6d1e8fbe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:54.066491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd056-d05c-415e-8b08-6b2c6d1e8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:54.083130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:54.140557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d27dd056-d05c-415e-8b08-6b2c6d1e8fbe None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:54.141776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d27dd056-d05c-415e-8b08-6b2c6d1e8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:54.239814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 678/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:17:54.528349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:17:54.529890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:54.587343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:54.634306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca09e2-38b6-4372-8434-5f75ebc6fe82 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:17:54.706906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:17:54.707510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:54.707813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:54.708085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:54.709672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:17:54.814084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:17:54.818227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:17:54.819688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dccba2-4f46-45c6-9e8a-d328d8c74ca9 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:17:54.839184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 671/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:17:59.787712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e045-ce9d-4ac8-9739-2d931b5cab48 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.788695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e045-ce9d-4ac8-9739-2d931b5cab48 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.789745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613e045-ce9d-4ac8-9739-2d931b5cab48 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.791932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 679/1350] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:17:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:17:59.798111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664409-7724-430e-8283-816224fd8944 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:17:59.798111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664409-7724-430e-8283-816224fd8944 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:17:59.798111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664409-7724-430e-8283-816224fd8944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:17:59.902745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6e664409-7724-430e-8283-816224fd8944 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:17:59.904304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e664409-7724-430e-8283-816224fd8944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:00.096166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 672/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:04.226838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5286-1572-44f1-9e38-39f6683e9165 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:04.227374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5286-1572-44f1-9e38-39f6683e9165 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:04.227374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5286-1572-44f1-9e38-39f6683e9165 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:04.228922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 680/1352] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:04.233639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602d57-1e78-487f-a687-65114ae85c29 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:04.234087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602d57-1e78-487f-a687-65114ae85c29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:04.234183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602d57-1e78-487f-a687-65114ae85c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:04.238051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=59 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:04.295472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:04.319492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8e602d57-1e78-487f-a687-65114ae85c29 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:04.321267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e602d57-1e78-487f-a687-65114ae85c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:04.431490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 673/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:05.838158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176371-70fd-4c1e-9476-d3b127c3a23a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:05.838707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176371-70fd-4c1e-9476-d3b127c3a23a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:05.838707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176371-70fd-4c1e-9476-d3b127c3a23a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:05.840107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 681/1354] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:05.845281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816222a-a7b3-4f2a-8f27-e52fd54b94d6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:05.845705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816222a-a7b3-4f2a-8f27-e52fd54b94d6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:05.845819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816222a-a7b3-4f2a-8f27-e52fd54b94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:05.924610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9816222a-a7b3-4f2a-8f27-e52fd54b94d6 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:05.925825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9816222a-a7b3-4f2a-8f27-e52fd54b94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:05.967497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:06.037226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 674/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:09.684809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d888d8-372c-479a-9ffc-5adad3351116 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:09.685384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d888d8-372c-479a-9ffc-5adad3351116 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:09.685384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d888d8-372c-479a-9ffc-5adad3351116 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:09.686966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 682/1356] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:09.692241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55839-2cf8-4dbd-aa92-aeed7d4cce6d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:09.692717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55839-2cf8-4dbd-aa92-aeed7d4cce6d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:09.692983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55839-2cf8-4dbd-aa92-aeed7d4cce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:09.764467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-27b55839-2cf8-4dbd-aa92-aeed7d4cce6d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:09.765927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-27b55839-2cf8-4dbd-aa92-aeed7d4cce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:09.868437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 675/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:10.172741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:10.173403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:10.173918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:10.175660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 683/1358] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 21:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:10.186789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b9c28-3206-444e-975e-47ee14d6becf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:10.187346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b9c28-3206-444e-975e-47ee14d6becf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:10.187985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b9c28-3206-444e-975e-47ee14d6becf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:10.206696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:10.270432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e50b9c28-3206-444e-975e-47ee14d6becf None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:10.271729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e50b9c28-3206-444e-975e-47ee14d6becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:10.394750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 676/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:10.403084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:10.404629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:10.564662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43c229-8c28-46a6-b4a8-2bc88cf42326 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:10.651321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:10.693713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:10.694369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:10.694703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:10.695471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:10.697129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:10.799073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:10.800705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b297f61-f817-4676-b434-02cd1b8327b7 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:10.827719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 684/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:13.226215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:13.226714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:13.227166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:13.229357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 677/1361] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:13.234655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea1ac-f43b-41c1-a031-7981c99190d0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:13.235199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea1ac-f43b-41c1-a031-7981c99190d0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:13.235199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea1ac-f43b-41c1-a031-7981c99190d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:13.326701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-547ea1ac-f43b-41c1-a031-7981c99190d0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:13.328077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-547ea1ac-f43b-41c1-a031-7981c99190d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:13.369869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:13.439693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 685/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:13.734891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:13.736933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:13.813772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:13.927773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b14d4-3ce0-4320-ae47-06a7bde0ae2d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:14.067507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:14.068141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:14.068415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:14.068679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:14.070521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:14.093619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:14.165091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:14.166751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b4dc8-5db7-4c66-8e9d-b375d473657d service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:14.186145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 678/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:16.705778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce454b-d302-419f-bca7-fc8925a6e006 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:16.706308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce454b-d302-419f-bca7-fc8925a6e006 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:16.706308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce454b-d302-419f-bca7-fc8925a6e006 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:16.707843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 686/1364] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:16.713143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b51c-d470-48ed-9f8a-f80ebfa119dc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:16.713567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b51c-d470-48ed-9f8a-f80ebfa119dc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:16.713737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b51c-d470-48ed-9f8a-f80ebfa119dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:16.782242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-71e3b51c-d470-48ed-9f8a-f80ebfa119dc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:16.783451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-71e3b51c-d470-48ed-9f8a-f80ebfa119dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:16.884127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:16.891916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 679/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:17.177417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:17.177701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:17.177975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:17.180600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 687/1366] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 21:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:17.186072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d83b-ffc9-409d-88b3-fc7554ec0cd0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:17.186361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d83b-ffc9-409d-88b3-fc7554ec0cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:17.186632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d83b-ffc9-409d-88b3-fc7554ec0cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:17.274019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d5d1d83b-ffc9-409d-88b3-fc7554ec0cd0 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:17.275247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d5d1d83b-ffc9-409d-88b3-fc7554ec0cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:17.404981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 680/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:17.413321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:17.415249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:17.573109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdcd00-3bb0-4faa-bd17-c404afeadf18 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:17.595154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:17.701609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:17.702131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:17.702406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:17.702673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:17.704319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:17.805963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:17.807586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41efd0c-71f2-42ab-886c-64ac7e93187c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:17.826302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 688/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:17.885765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921395-0776-4385-a98e-16330a502c01 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:17.886296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921395-0776-4385-a98e-16330a502c01 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:17.886296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921395-0776-4385-a98e-16330a502c01 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:17.887802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 681/1369] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:17.893181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e97bd6-d4c0-4e24-a818-2df874024615 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:17.893472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e97bd6-d4c0-4e24-a818-2df874024615 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:17.893747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e97bd6-d4c0-4e24-a818-2df874024615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:17.938785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:17.975874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b0e97bd6-d4c0-4e24-a818-2df874024615 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:17.977057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0e97bd6-d4c0-4e24-a818-2df874024615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:18.097691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 689/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:18.429151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f953-e437-4594-965b-43268df78f16 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:18.429151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f953-e437-4594-965b-43268df78f16 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:18.435393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f953-e437-4594-965b-43268df78f16 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:18.435393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 682/1371] 10.222.0.46 () {54 vars in 964 bytes} [Fri Sep 9 21:18:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:18.440651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f86a-32ff-4e54-8a83-9dec10e40aa8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:18.441059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f86a-32ff-4e54-8a83-9dec10e40aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:18.441459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494f86a-32ff-4e54-8a83-9dec10e40aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:18.528621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0494f86a-32ff-4e54-8a83-9dec10e40aa8 None None] MFA Rules not processed for user `b670a7afd2264609894dec2feabc4581`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:18.530108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0494f86a-32ff-4e54-8a83-9dec10e40aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:18.581211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:18.614549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 690/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Fri Sep 9 21:18:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:18.623155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f953-e437-4594-965b-43268df78f16 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:18.627135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7a81f953-e437-4594-965b-43268df78f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:18.679293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:18.736943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f953-e437-4594-965b-43268df78f16 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:18.832221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:18.833696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:18.833696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:18.833696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:18.836750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:18.952023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:18.969358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:18.971321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81bffe-cc5d-4ea6-84e7-2c434cc0d232 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:19.006898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 683/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:20.075941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:20.076510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:20.076510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:20.077859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 691/1374] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:20.083162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f010eb-1c04-4fcd-b408-d26fd4690c6c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:20.083591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f010eb-1c04-4fcd-b408-d26fd4690c6c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:20.083867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f010eb-1c04-4fcd-b408-d26fd4690c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:20.150614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-37f010eb-1c04-4fcd-b408-d26fd4690c6c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:20.151839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-37f010eb-1c04-4fcd-b408-d26fd4690c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:20.248061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 684/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:20.548172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:20.549945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:20.715145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c95b-c38e-4fe0-a4bd-a90994a1d520 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:20.770304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:20.836695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:20.837307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:20.837587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:20.837860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:20.839492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:20.941731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:20.943345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8afee-34f0-43cf-9fdc-f5d8d8681332 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:20.963662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 692/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:23.208412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:23.208983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:23.208983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:23.210609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 685/1377] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:23.215987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346e4e-ef72-4ec2-bf59-d19d3cef6d3b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:23.216444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346e4e-ef72-4ec2-bf59-d19d3cef6d3b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:23.216540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346e4e-ef72-4ec2-bf59-d19d3cef6d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:23.291946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-71346e4e-ef72-4ec2-bf59-d19d3cef6d3b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:23.293678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-71346e4e-ef72-4ec2-bf59-d19d3cef6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:23.301193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:23.421078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 693/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:23.706246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:23.707773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:23.775654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:23.865471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfda193-f383-41ef-bac7-4962a1cfd9b8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:23.981024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:23.981625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:23.981902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:23.982170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:23.984377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:24.006072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:24.082584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:24.084214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9599d67-e3fb-44ec-9202-8d15707b7736 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:24.102876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 686/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:26.328856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:26.329423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:26.329423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:26.330991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 694/1380] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:26.336343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3a43-1e51-4620-89f0-bd3965e54b34 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:26.336780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3a43-1e51-4620-89f0-bd3965e54b34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:26.337120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3a43-1e51-4620-89f0-bd3965e54b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:26.408516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c30b3a43-1e51-4620-89f0-bd3965e54b34 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:26.409771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c30b3a43-1e51-4620-89f0-bd3965e54b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:26.534079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:26.547764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 687/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:26.849562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:26.851388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:27.023729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:27.036187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ac3b-043a-422d-9b40-7840e66a389e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:27.167159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:27.167797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:27.168090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:27.168353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:27.170003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:27.279647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:27.281280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b273-69ad-4475-996e-ae7eb5580583 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:27.292885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:27.308316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 695/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:18:30.124124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:30.124696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:30.124696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:30.125969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 688/1383] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:30.133588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a85f-9f0b-464d-9af2-35d1a81310d6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:30.134115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a85f-9f0b-464d-9af2-35d1a81310d6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:30.134272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a85f-9f0b-464d-9af2-35d1a81310d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:30.219214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-80a7a85f-9f0b-464d-9af2-35d1a81310d6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:30.220938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80a7a85f-9f0b-464d-9af2-35d1a81310d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:30.344401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 696/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:30.692243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:30.694151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:30.871442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514c962-40e9-48e9-9426-3ae517ec7dd1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:30.950622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:30.998474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:30.999092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:30.999374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:30.999684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:31.001351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:31.099731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:31.102090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9b60-76db-48a3-b9d1-8515b76319d9 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:31.120830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 689/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:33.333032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:33.333632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:33.333632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:33.334934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 697/1386] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:33.341165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a48597-9ff7-4400-aa6c-10b2586a5b09 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:33.341640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a48597-9ff7-4400-aa6c-10b2586a5b09 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:33.341886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a48597-9ff7-4400-aa6c-10b2586a5b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:33.412133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-76a48597-9ff7-4400-aa6c-10b2586a5b09 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:33.413356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-76a48597-9ff7-4400-aa6c-10b2586a5b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:33.448227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:33.517780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 690/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:33.808918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:33.810537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:33.890385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:33.960930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e26f4a-4942-4366-a01d-e4b8009fe6ea None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:34.075875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:34.076489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:34.076774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:34.077045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:34.078677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:34.121913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:34.173824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:34.175445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c643f0d-2625-405c-aa7f-08bbc2a4fce0 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:34.194034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 698/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:34.336408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dabd32-b698-40e5-bcda-a9fe2a4585e6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:34.336693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dabd32-b698-40e5-bcda-a9fe2a4585e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:34.337695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dabd32-b698-40e5-bcda-a9fe2a4585e6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:34.339216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 691/1389] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:34.345208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92267-737d-4f97-9ce0-c82f295918bc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:34.345795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92267-737d-4f97-9ce0-c82f295918bc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:34.345795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92267-737d-4f97-9ce0-c82f295918bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:34.423493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a5c92267-737d-4f97-9ce0-c82f295918bc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:34.424911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5c92267-737d-4f97-9ce0-c82f295918bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:34.525533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 699/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:34.824391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:34.824854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:34.824964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:34.826274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 692/1391] 10.222.0.46 () {54 vars in 966 bytes} [Fri Sep 9 21:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:34.831387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a7536-f99e-4ea2-9900-f011328dd617 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:34.831764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a7536-f99e-4ea2-9900-f011328dd617 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:34.832104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a7536-f99e-4ea2-9900-f011328dd617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:34.848771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:34.911238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-890a7536-f99e-4ea2-9900-f011328dd617 None None] MFA Rules not processed for user `8985883a056c431b885bb07303d3f8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:34.912448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-890a7536-f99e-4ea2-9900-f011328dd617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:35.000612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 700/1392] 10.222.0.46 () {58 vars in 1063 bytes} [Fri Sep 9 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:35.009918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:35.011662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:35.118232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116677c-ee7a-4a52-837d-f317976f73aa None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:35.179718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:35.195965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:35.197477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:35.197932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:35.198446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:35.200392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:35.312865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:35.314312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dab665-f992-4f63-954f-37acaef6f8cf service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:35.333065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 693/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:36.092907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503e0b-ffd3-41ec-ad34-ad21b2de4082 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.093435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503e0b-ffd3-41ec-ad34-ad21b2de4082 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.093435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc503e0b-ffd3-41ec-ad34-ad21b2de4082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.161793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc503e0b-ffd3-41ec-ad34-ad21b2de4082 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:36.163016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc503e0b-ffd3-41ec-ad34-ad21b2de4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:36.217149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:36.249450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 701/1394] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:36.298950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb205d-6a8d-437b-a659-5b879de10786 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.299602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb205d-6a8d-437b-a659-5b879de10786 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.299602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb205d-6a8d-437b-a659-5b879de10786 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.300942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 694/1395] 10.222.0.46 () {54 vars in 943 bytes} [Fri Sep 9 21:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:36.308285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.308575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.308849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.387724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] MFA Rules not processed for user `103b5c20d7354245a2c57adb0363ffce`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:36.389180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:36.416310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19a253-cf08-48d7-bd94-fb83b105eeb7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.416607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19a253-cf08-48d7-bd94-fb83b105eeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.416878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19a253-cf08-48d7-bd94-fb83b105eeb7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.418213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 695/1396] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:36.423249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.423551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.423840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.489989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:36.491236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:36.501897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 702/1397] 10.222.0.46 () {58 vars in 1040 bytes} [Fri Sep 9 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:36.512053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:36.522411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9695a-7e7b-4857-8ded-16b65b154045 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:36.534520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:36.549112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:36.593389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 696/1398] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:18:36.636742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:36.637430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:36.637710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:36.637979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:36.639926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:36.730919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:36.733335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6196-0784-4700-bf05-b27728c03018 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:36.741312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 703/1399] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:18:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 21:18:36.890661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:36.892068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:37.030282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f932584-ef78-4e7f-8976-35d6cc633ef8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:37.095766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:37.160593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:37.161200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:37.161477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:37.161748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:37.163365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:37.270545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:37.272180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0829ee2-9770-4db6-84a2-63b1d3d25db9 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:37.298599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 697/1400] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:39.532170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:39.532729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:39.532729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:39.534057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 704/1401] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:39.539386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b66bc-7798-457f-907f-e113038195ba None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:39.540455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b66bc-7798-457f-907f-e113038195ba None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:39.540620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b66bc-7798-457f-907f-e113038195ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:39.615110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0e3b66bc-7798-457f-907f-e113038195ba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:39.616380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0e3b66bc-7798-457f-907f-e113038195ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:39.638212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:39.733149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 698/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:40.024500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:40.026199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:40.104521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:40.201592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e50a12-3276-481c-a524-4659c4d6420b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:40.334706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:40.335225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:40.335503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:40.335801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:40.338262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:40.358227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:40.442453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:40.444239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ee64a-bfed-4723-a9c9-e7afef5a61b1 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:40.465762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 705/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:42.723469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:42.724031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:42.724031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:42.725391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 699/1404] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:42.731029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfd1f-e234-483d-b353-21400f4f3d24 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:42.731489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfd1f-e234-483d-b353-21400f4f3d24 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:42.731758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dfd1f-e234-483d-b353-21400f4f3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:42.808160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-011dfd1f-e234-483d-b353-21400f4f3d24 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:42.809540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-011dfd1f-e234-483d-b353-21400f4f3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:42.913923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:42.921423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 706/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:43.213554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:43.215123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:43.379286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:43.382398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1468-c3fe-49d7-9c06-0347d86325ff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:43.511128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:43.511677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:43.511962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:43.512235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:43.513875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:43.614095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:43.615822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0d1cc-e0d3-47d7-a7e9-21d39fa95a3e service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:43.641044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 700/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:45.871446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:45.872010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:45.872010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:45.873371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 707/1407] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:45.878740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1b34-6421-4e77-afa3-b3f981bace94 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:45.879155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1b34-6421-4e77-afa3-b3f981bace94 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:45.879335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1b34-6421-4e77-afa3-b3f981bace94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:45.884071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:46.082278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b42c1b34-6421-4e77-afa3-b3f981bace94 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:46.083912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b42c1b34-6421-4e77-afa3-b3f981bace94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.194687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 701/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:46.489472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:18:46.491327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.654829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9778f85-2640-43d4-9c7d-11a11f18cc88 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:18:46.726572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:46.782035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:18:46.782951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:46.783225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:46.783491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:46.785443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:46.886551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:18:46.888195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f9ac3-d29d-4d20-a661-f1db0e5f7e03 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:18:46.907970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 708/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:18:56.192662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=64 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:56.905507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=64 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:59.518146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97f6f-8c70-4681-b16d-222e3e21ad7a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:59.518709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97f6f-8c70-4681-b16d-222e3e21ad7a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:59.518709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97f6f-8c70-4681-b16d-222e3e21ad7a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:59.520058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 702/1410] 10.222.0.46 () {54 vars in 918 bytes} [Fri Sep 9 21:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:59.525092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ab7a-388f-4089-b2ca-980ad832fa9b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:59.525533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ab7a-388f-4089-b2ca-980ad832fa9b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:59.525636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ab7a-388f-4089-b2ca-980ad832fa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:59.602213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2bb4ab7a-388f-4089-b2ca-980ad832fa9b None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:59.603448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2bb4ab7a-388f-4089-b2ca-980ad832fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:59.709739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 709/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Sep 9 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:18:59.725297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215bf0b-2cf7-47c9-b88a-341e31bea9e7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:59.725580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215bf0b-2cf7-47c9-b88a-341e31bea9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:59.725851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215bf0b-2cf7-47c9-b88a-341e31bea9e7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:59.727154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 703/1412] 10.222.0.46 () {54 vars in 918 bytes} [Fri Sep 9 21:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:18:59.732201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ac9d-e6a4-4240-b348-01c74c3594b3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:18:59.732481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ac9d-e6a4-4240-b348-01c74c3594b3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:18:59.732750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ac9d-e6a4-4240-b348-01c74c3594b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:18:59.804719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:18:59.808008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-06f8ac9d-e6a4-4240-b348-01c74c3594b3 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:18:59.809889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-06f8ac9d-e6a4-4240-b348-01c74c3594b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:18:59.925236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 710/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Sep 9 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:02.524844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:02.525421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:02.525421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:02.527412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:19:02.532751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ac9e-c835-46c6-8e71-84a629edcfc3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:02.533165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ac9e-c835-46c6-8e71-84a629edcfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:02.533306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ac9e-c835-46c6-8e71-84a629edcfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:02.600322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a55ac9e-c835-46c6-8e71-84a629edcfc3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:02.601520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a55ac9e-c835-46c6-8e71-84a629edcfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:02.657800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:02.699978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:02.987070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:02.988640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:03.132160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a70733-86bd-45f5-9f40-7334f94dbd8a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:03.205274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:03.243921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:03.244450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:03.244722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:03.245553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:03.247272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:03.336473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:03.338057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cf644-157b-411c-9ba8-773c04615d41 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:03.356908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 705/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:19:04.539705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:04.540290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:04.540290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:04.541653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:04.548609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d77a5-5ed4-43f5-bb22-2debd5a9f5cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:04.549062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d77a5-5ed4-43f5-bb22-2debd5a9f5cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:04.549160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d77a5-5ed4-43f5-bb22-2debd5a9f5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:04.624824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-409d77a5-5ed4-43f5-bb22-2debd5a9f5cd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:04.626530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-409d77a5-5ed4-43f5-bb22-2debd5a9f5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:04.660818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:04.731177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:04.881771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:04.883231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:05.034117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ec765-4bb9-4a09-8cc1-79148b360594 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:05.075086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:05.150753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:05.151792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:05.152151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:05.152596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:05.154205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:05.242631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:05.244254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b32c2-883e-4ca1-bf65-c6413a98eff0 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:05.265132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 713/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:06.406703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:06.407239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:06.407239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:06.408531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:06.414642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ff38-ea03-4f69-82f3-7d31a021981d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:06.415063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ff38-ea03-4f69-82f3-7d31a021981d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:06.415252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ff38-ea03-4f69-82f3-7d31a021981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:06.471723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:06.493519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-54c4ff38-ea03-4f69-82f3-7d31a021981d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:06.494736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-54c4ff38-ea03-4f69-82f3-7d31a021981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:06.591502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:06.749460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:06.750986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:06.892878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75b74-e46e-48d2-8a27-66eac13015ff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:06.905242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:07.006404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:07.006924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:07.007203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:07.007473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:07.009196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:07.099391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:07.101001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59d5e9-2aad-422d-a37c-38c329006bcd service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:07.118626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 708/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:08.243323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:08.243953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:08.243953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:08.245284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:08.251658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc25df4-7f58-4c3b-bedf-50d8f59b9143 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:08.252137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc25df4-7f58-4c3b-bedf-50d8f59b9143 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:08.252235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc25df4-7f58-4c3b-bedf-50d8f59b9143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:08.273445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:08.330157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bbc25df4-7f58-4c3b-bedf-50d8f59b9143 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:08.331417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bbc25df4-7f58-4c3b-bedf-50d8f59b9143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:08.428912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:08.576910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:08.578351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:08.689951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:08.725702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1edb07-dca1-445b-ae6f-608e45d10807 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:08.840528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:08.841061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:08.841340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:08.841617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:08.843238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:08.924145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:08.938462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:08.940120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48260238-f579-41ec-8416-e34162e15be5 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:08.958120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 716/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:10.086722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:10.087270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:10.087270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:10.088718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:10.096565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76057b-466c-4f92-b275-bb2819a5e96c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:10.096975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76057b-466c-4f92-b275-bb2819a5e96c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:10.097158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76057b-466c-4f92-b275-bb2819a5e96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:10.174868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d76057b-466c-4f92-b275-bb2819a5e96c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:10.176103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d76057b-466c-4f92-b275-bb2819a5e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:10.272036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:19:10.423617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:10.425123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:10.509196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:10.571441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503d243-42dc-4e46-a993-6b1295e23362 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:10.685658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:10.686169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:10.686449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:10.686708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:10.688334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:10.735678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:10.777865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:10.779452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef0068-8fbe-4034-93d1-2ac67efd2601 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:10.797838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 711/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:11.871262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:11.871837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:11.871837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:11.873190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:11.879579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629297f5-3d5a-437d-b5ec-b1545e0b6e8f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:11.880010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629297f5-3d5a-437d-b5ec-b1545e0b6e8f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:11.880126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629297f5-3d5a-437d-b5ec-b1545e0b6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:11.958461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-629297f5-3d5a-437d-b5ec-b1545e0b6e8f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:11.959626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-629297f5-3d5a-437d-b5ec-b1545e0b6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:12.054948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:12.208179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:12.209759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:12.258602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:12.357341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e97f3-d59b-442a-81c3-35df4d669804 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:12.470107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:12.470640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:12.470920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:12.471193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:12.472837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:12.479713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:12.564626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:12.566232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334d83-b782-4ffa-bd8b-62c42d201c39 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:12.583484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 719/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:13.641137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:13.641628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:13.641727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:13.643282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:13.649580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c604c5-dba3-4875-b2be-e2722b4d9cb9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:13.649991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c604c5-dba3-4875-b2be-e2722b4d9cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:13.650117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c604c5-dba3-4875-b2be-e2722b4d9cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:13.725283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a9c604c5-dba3-4875-b2be-e2722b4d9cb9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:13.726398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9c604c5-dba3-4875-b2be-e2722b4d9cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:13.793275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:13.821990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:13.971003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:13.972584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:13.990433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:14.120294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e17e96-fe01-4ac1-9f37-55ae1251f5b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:14.208541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:14.234344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:14.234944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:14.235225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:14.235492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:14.237130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:14.326418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:14.327958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a769-9ce9-45f6-964e-a708b8da6935 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:14.346075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 714/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:15.413731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:15.414257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:15.414374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:15.415718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:15.422093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b727c-0535-4274-b6f6-a7bd33aef8cb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:15.422544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b727c-0535-4274-b6f6-a7bd33aef8cb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:15.422650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b727c-0535-4274-b6f6-a7bd33aef8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:15.497553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b72b727c-0535-4274-b6f6-a7bd33aef8cb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:15.498762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b72b727c-0535-4274-b6f6-a7bd33aef8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:15.540823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:15.603195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:15.752528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:15.754065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:15.900244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd450-c6e2-47cd-9898-a4a5f2e4a059 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:15.954346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:16.013827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:16.014437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:16.014719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:16.014994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:16.016660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:16.115730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:16.117378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b286507-c46b-4408-993f-17e56f713461 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:16.136068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 722/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:17.198144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:17.198688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:17.198688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:17.200198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:17.206605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c3edf-8e25-43d2-90e9-fc445e9b6353 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:17.207070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c3edf-8e25-43d2-90e9-fc445e9b6353 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:17.207170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c3edf-8e25-43d2-90e9-fc445e9b6353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:17.284333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ff9c3edf-8e25-43d2-90e9-fc445e9b6353 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:17.285556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ff9c3edf-8e25-43d2-90e9-fc445e9b6353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:17.287579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:17.382479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:17.533972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:17.535459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:17.684182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f39a-4fea-42fb-9b9b-96e6d9c1eea8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:17.710767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:17.795341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:17.795895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:17.796168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:17.796435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:17.798025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:17.885505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:17.887167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88530ed-4d3b-47dc-bbd1-9d1cce4c0b51 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:17.905407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 717/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:20.010236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:20.011021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:20.011021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:20.012617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:20.018532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ceb2b-f205-4c6e-bf20-f452aab7dece None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:20.018988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ceb2b-f205-4c6e-bf20-f452aab7dece None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:20.019091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ceb2b-f205-4c6e-bf20-f452aab7dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:20.069375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:20.094537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-326ceb2b-f205-4c6e-bf20-f452aab7dece None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:20.095938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-326ceb2b-f205-4c6e-bf20-f452aab7dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:20.192205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:20.480773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:20.482327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:20.586943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e98fe-f865-45d1-a41a-0d0477ef6488 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:20.618599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:20.664579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:20.665181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:20.665456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:20.665723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:20.667332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:20.776201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:20.778219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03a1d7-f4a1-457a-a4c5-f91740d0d243 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:20.796723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 725/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:19:23.185050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:23.185616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:23.185616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:23.186924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:23.192249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ede05-2a39-4c9b-ba1a-c5544c0f5833 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:23.192693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ede05-2a39-4c9b-ba1a-c5544c0f5833 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:23.192817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ede05-2a39-4c9b-ba1a-c5544c0f5833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:23.258882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5c0ede05-2a39-4c9b-ba1a-c5544c0f5833 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:23.260115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c0ede05-2a39-4c9b-ba1a-c5544c0f5833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:23.273069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:23.356833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:23.651490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:23.653855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:23.755908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907f9d4-5960-43bf-b55e-3f066300da70 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:23.769391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:23.829494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:23.830086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:23.830359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:23.830616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:23.832236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:23.938287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:23.939737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21937d52-c265-44bc-8e1c-6d8a3f4f0a3b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:23.957718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 720/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:26.443016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:26.443616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:26.443616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:26.444914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:26.450964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15bbc5-4989-46d7-9864-59e38ce35c74 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:26.451421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15bbc5-4989-46d7-9864-59e38ce35c74 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:26.451519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15bbc5-4989-46d7-9864-59e38ce35c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:26.499912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:26.518311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4b15bbc5-4989-46d7-9864-59e38ce35c74 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:26.519792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4b15bbc5-4989-46d7-9864-59e38ce35c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:26.613494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:26.903551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:26.905178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:27.007798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958c125-6f5f-4394-a25a-66d694f9b963 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:27.050741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:27.085978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:27.086580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:27.086863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:27.087139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:27.088784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:27.198424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:27.200663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cb404-57a3-44ff-b8af-f5e37c321509 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:27.218515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 728/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:29.773043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:29.773555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:29.773555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:29.774839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:29.780747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021f5ae-b5ba-48f8-b59d-2d93d2d60d9b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:29.781181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021f5ae-b5ba-48f8-b59d-2d93d2d60d9b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:29.781280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021f5ae-b5ba-48f8-b59d-2d93d2d60d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:29.849093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1021f5ae-b5ba-48f8-b59d-2d93d2d60d9b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:29.850325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1021f5ae-b5ba-48f8-b59d-2d93d2d60d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:29.871291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:29.948325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:30.239628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:30.241165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:30.345504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7b14e-1ba4-44dc-ac6b-ce9c0d1e5803 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:30.366446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:30.419237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:30.419893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:30.420175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:30.420450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:30.422068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:30.531917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:30.533252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606f401-f712-440c-a59a-d6634a2a35dc service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:30.550797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 723/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:32.761279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:32.761834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:32.761834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:32.763823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:32.769083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b240-b527-4d92-a239-439ff80d1718 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:32.769775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b240-b527-4d92-a239-439ff80d1718 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:32.769872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b240-b527-4d92-a239-439ff80d1718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:32.832613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:32.835418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-64f1b240-b527-4d92-a239-439ff80d1718 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:32.837224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-64f1b240-b527-4d92-a239-439ff80d1718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:32.934966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:19:33.223430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:33.225011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:33.329631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba6b3-1f00-4f4c-a912-20ee6042ac59 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:33.379522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:33.404737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:33.405343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:33.405927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:33.406207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:33.407831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:33.520315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:33.521678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f39d0-4d0c-470e-8046-864909720827 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:33.539887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 731/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:35.755180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:35.755728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:35.755728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:35.757004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:35.762322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2cbc-b8aa-493c-b454-6845eb1a41c7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:35.762726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2cbc-b8aa-493c-b454-6845eb1a41c7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:35.763499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2cbc-b8aa-493c-b454-6845eb1a41c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:35.827671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e51f2cbc-b8aa-493c-b454-6845eb1a41c7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:35.828867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e51f2cbc-b8aa-493c-b454-6845eb1a41c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:35.856830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:35.925555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:36.209343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:36.210896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:36.314479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b98cb-7203-4ea3-a70c-5dcd7f1ce22d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:36.343694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:36.395365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:36.395998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:36.396276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:36.396541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:36.398132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:36.506078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:36.507422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff967084-7800-4012-8231-6d671842f413 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:36.526442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 726/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:38.762909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:38.763471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:38.763880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:38.765890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:38.771352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfdbec-707a-4144-8703-e81d1c9d7fff None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:38.771813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfdbec-707a-4144-8703-e81d1c9d7fff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:38.771923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfdbec-707a-4144-8703-e81d1c9d7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:38.836972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-54dfdbec-707a-4144-8703-e81d1c9d7fff None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:38.838181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54dfdbec-707a-4144-8703-e81d1c9d7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:38.846759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:38.940368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:39.247186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:39.248825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:39.354170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fef36e-0770-416d-877a-a23f3e402e14 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:39.410173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:39.429439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:39.430057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:39.430338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:39.430616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:39.432265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:39.542595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:39.544013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c3e8f-9f39-4e8c-8b64-8d683d3396d8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:39.562419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 734/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:19:41.945053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:41.945629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:41.945629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:41.946993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:41.953277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f605-ab96-4467-882d-4d15d96781e1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:41.953724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f605-ab96-4467-882d-4d15d96781e1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:41.953839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f605-ab96-4467-882d-4d15d96781e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:42.022307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b310f605-ab96-4467-882d-4d15d96781e1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:42.023460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b310f605-ab96-4467-882d-4d15d96781e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:42.062258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:42.130790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:42.458167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:42.460672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:42.569943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e565c-b818-4b71-a8f9-9b6f3050f1a1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:42.616730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:42.655818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:42.656426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:42.656707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:42.656979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:42.658622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:42.770169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:42.771804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1361d3-98a4-4089-aaf8-1068ff832427 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:42.789552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 729/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:45.002694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:45.003260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:45.003260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:45.004651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:45.010094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d801-a377-4e5b-930b-e80137edd19f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:45.010469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d801-a377-4e5b-930b-e80137edd19f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:45.010876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d801-a377-4e5b-930b-e80137edd19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:45.079582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0af4d801-a377-4e5b-930b-e80137edd19f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:45.080800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0af4d801-a377-4e5b-930b-e80137edd19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:45.098838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:45.176445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:45.463579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:45.465162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:45.584997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ed1e5-7525-46b7-a64c-197bca84a859 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:45.652864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:45.665829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:45.666794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:45.667319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:45.667756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:45.670311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:45.782659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:45.784054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3a458-d3e6-40b9-ad91-83dcc10a3d9b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:45.803584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 737/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:47.991171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:47.991776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:47.991776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:47.993112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:19:47.998455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab9e5-ac1d-4cd1-bf23-d7cd3c9935a9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:47.998862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab9e5-ac1d-4cd1-bf23-d7cd3c9935a9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:47.998999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ab9e5-ac1d-4cd1-bf23-d7cd3c9935a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:48.067384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6f6ab9e5-ac1d-4cd1-bf23-d7cd3c9935a9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:19:48.068725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f6ab9e5-ac1d-4cd1-bf23-d7cd3c9935a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:48.117345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:48.165967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:19:48.455753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:19:48.457290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:48.562108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f684ab0-d3db-4995-9609-3243f62af4b6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:19:48.610035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:48.636016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:19:48.636776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:19:48.636917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:19:48.637182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:19:48.638825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:19:48.747729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:19:48.749497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87037f85-cebc-400a-86bc-da5b0c819a22 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:19:48.767822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 732/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:19:58.164489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=22 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:19:58.765551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=66 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:08.885634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bff8e-58a4-43bb-8940-f68c0c3785c5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:08.886249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bff8e-58a4-43bb-8940-f68c0c3785c5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:08.886249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bff8e-58a4-43bb-8940-f68c0c3785c5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:08.887594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 739/1471] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:08.893733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6831f9-aed8-42cd-83bf-f299602111fa None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:08.894182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6831f9-aed8-42cd-83bf-f299602111fa None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:08.894285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6831f9-aed8-42cd-83bf-f299602111fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:08.960545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f6831f9-aed8-42cd-83bf-f299602111fa None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:08.961761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f6831f9-aed8-42cd-83bf-f299602111fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:09.058364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 733/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:09.362232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:09.362511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:09.362805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:09.364157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 740/1473] 10.222.0.46 () {54 vars in 966 bytes} [Fri Sep 9 21:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:09.369276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08fad-76e4-4cf8-8d91-03236779d9d8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:09.369558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08fad-76e4-4cf8-8d91-03236779d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:09.369827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08fad-76e4-4cf8-8d91-03236779d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:09.447415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-73b08fad-76e4-4cf8-8d91-03236779d9d8 None None] MFA Rules not processed for user `8985883a056c431b885bb07303d3f8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:09.448647 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-73b08fad-76e4-4cf8-8d91-03236779d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:09.469613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:09.525193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 734/1474] 10.222.0.46 () {58 vars in 1063 bytes} [Fri Sep 9 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:09.533366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:09.534942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:09.641532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2bd9c-27cc-4537-a1e6-cb8b616d0da3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:09.716934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:09.717534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:09.717860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:09.718137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:09.719770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:09.781329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:09.836386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:09.837814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad5d4d-71bb-4089-8e5d-1fbd2c13f8b0 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:09.856914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 741/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:12.099348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:12.099911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:12.099911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:12.101235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 735/1476] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:12.106366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddaef0-2593-4770-ab5e-e360662d955d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:12.106776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddaef0-2593-4770-ab5e-e360662d955d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:12.106912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddaef0-2593-4770-ab5e-e360662d955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:12.172197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-beddaef0-2593-4770-ab5e-e360662d955d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:12.173392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-beddaef0-2593-4770-ab5e-e360662d955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:12.267929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 742/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:12.544865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:12.546512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:12.671832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825772f-7379-40ca-ba1f-5bec39778525 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:12.729597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:12.775076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:12.775718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:12.775995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:12.776265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:12.776901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:12.778630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:20:12.778630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:20:12.778630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:20:12.779784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ee91-4445-43a0-b549-a82c379383db admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:12.799423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 736/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 21:20:12 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:20:15.006851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:15.007395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:15.007519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:15.008849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 743/1479] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:20:15.014385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb19af-0f3e-4534-b9e4-7a8e9e0b2638 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:15.014860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb19af-0f3e-4534-b9e4-7a8e9e0b2638 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:15.014860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb19af-0f3e-4534-b9e4-7a8e9e0b2638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:15.080435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fecb19af-0f3e-4534-b9e4-7a8e9e0b2638 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:15.081644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fecb19af-0f3e-4534-b9e4-7a8e9e0b2638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:15.176294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 737/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:15.469900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:15.471456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:15.481198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:15.617247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea489871-4938-48a1-a434-fcb672038c54 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:15.700141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:15.734539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:15.735089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:15.735373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:15.735689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:15.737315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:15.831089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:15.832731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c68ea-1641-4512-bf20-8d0ecd2f9104 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:15.850863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 744/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:20:18.200399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:18.200968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:18.200968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:18.202294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 738/1482] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:18.207744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54a33-44e8-42ec-b263-af0bc4ce4156 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:18.208189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54a33-44e8-42ec-b263-af0bc4ce4156 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:18.208317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54a33-44e8-42ec-b263-af0bc4ce4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:18.274256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c1c54a33-44e8-42ec-b263-af0bc4ce4156 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:18.275444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c1c54a33-44e8-42ec-b263-af0bc4ce4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:18.320509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:18.370704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 745/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:18.661822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:18.663381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:18.688031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:18.766958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43c1be-e2a2-4c44-8567-86f118fb5466 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:18.839448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:18.840081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:18.840355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:18.840623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:18.842205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:18.915516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:18.950880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:18.952334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861565aa-13d4-4cba-a441-5132b47ce7bc service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:18.971511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 739/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:21.176095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:21.176693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:21.176693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:21.177974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 746/1485] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:21.183318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8830c5c-5d08-4674-8db0-1ea35d16468c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:21.183890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8830c5c-5d08-4674-8db0-1ea35d16468c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:21.184167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8830c5c-5d08-4674-8db0-1ea35d16468c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:21.250531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d8830c5c-5d08-4674-8db0-1ea35d16468c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:21.251771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d8830c5c-5d08-4674-8db0-1ea35d16468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:21.348969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 740/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:20:21.654306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:21.656398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:21.769947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cd3b2-b2d3-4198-8190-78febc59dec6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:21.849674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:21.851279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:21.852012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:21.852479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:21.853000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:21.854847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:21.962372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:21.963837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31da3a5-dddd-4e5d-89d1-8abe857ace69 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:21.982644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 747/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:24.317480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:24.318045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:24.318045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:24.319485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 741/1488] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:24.324862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07203f-b9df-4f65-ae2d-60dcacb1a4ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:24.325294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07203f-b9df-4f65-ae2d-60dcacb1a4ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:24.325398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07203f-b9df-4f65-ae2d-60dcacb1a4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:24.509160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d07203f-b9df-4f65-ae2d-60dcacb1a4ba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:24.510384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d07203f-b9df-4f65-ae2d-60dcacb1a4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:24.563258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:24.606164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 748/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:24.898869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:24.900454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:24.936299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:25.003995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21caa4b-c113-422b-8871-26d6f9266cb4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:25.079341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:25.079968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:25.080242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:25.080514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:25.082104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:25.164158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:25.193679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:25.195109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ad1d-81f8-45fb-8263-576e9e8a9b8a service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:25.212996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 742/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:27.622304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:27.622876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:27.622876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:27.624256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 749/1491] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 21:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:20:27.630629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d2de3-4ff0-4e10-9dc1-8381bb1de607 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:27.631114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d2de3-4ff0-4e10-9dc1-8381bb1de607 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:27.631213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d2de3-4ff0-4e10-9dc1-8381bb1de607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:27.698423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-970d2de3-4ff0-4e10-9dc1-8381bb1de607 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:20:27.699663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-970d2de3-4ff0-4e10-9dc1-8381bb1de607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:27.800767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 743/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:20:28.097848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:20:28.099390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:28.201234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c92b73-4277-4430-a14a-db82ad84ddf8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:20:28.275350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:20:28.275972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:20:28.276248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:20:28.276515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:20:28.278100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:20:28.287780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:28.391931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:20:28.393279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee0daa-d65c-4ffb-91b7-49d18e15f0dd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:20:28.411669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 750/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:20:37.799232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=82 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:20:38.409389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=50 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:41.604343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:41.604888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:41.604888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:41.680830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:41.681967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:41.779177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 744/1494] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:41.787342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:41.787854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:41.787959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:41.866266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:41.867465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:41.963753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 751/1495] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:41.968292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:41.969490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:41.980938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1d60-4bd5-4b4a-8c1f-20dbd1b430fc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.043677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:42.078003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.078608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.078886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.079155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.079830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:21:42.080768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.082112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.082112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:21:42.082112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:21:42.083013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.093990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 745/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:21:42.103001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:42.111940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c413c9b-c46c-41a3-8b9b-78b88f217695 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.176756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:42.213535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.214200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.214403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.214670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.215295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.217063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.217063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:21:42.217063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:21:42.218225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.273211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 752/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:21:42 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:21:42.277386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:42.279205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:42.312880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc74be-5d74-48a0-9bba-7c0a86f74c1a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.406223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.406830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.407104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.407368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.408025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.409354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.409354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:21:42.409354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:21:42.410550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.451808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 746/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:21:42 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:21:42.456438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:42.457959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:42.517279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16762eda-b27e-4cba-b6a8-2a5af66b1424 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.601357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:42.613133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.613736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.614014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.614285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.614926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.615978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.615978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:21:42.615978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:21:42.616854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.629753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 753/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:21:42.633778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:42.635349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:42.647762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f2ba-1483-4611-952b-532518e4e80c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.673545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:42.744447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.745048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.745318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.745584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.746218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.747213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.747213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:21:42.747213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:21:42.748119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad3911-b173-4e26-a440-499a79fa9bbc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.760749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 747/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:21:42.764937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:42.766445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:42.799390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901eaf3-1522-4e3f-a2bb-30dfe1b0f40a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:42.891424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:42.892100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.892383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.892846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] PATH_INFO: `/v3/projects/c5b8d0df147f4a8980d3bd48f5a7c786/users/9c4961b40e044ab9b85be34bb1e97bf6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:42.915412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b8d0df147f4a8980d3bd48f5a7c786, user_id=9c4961b40e044ab9b85be34bb1e97bf6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:42.916473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:42.916473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:21:42.916473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:21:42.918371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e615527-26ea-4afa-b6f6-f0ec34b4dc28 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:42.942973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 754/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:21:42 2022] PUT /identity/v3/projects/c5b8d0df147f4a8980d3bd48f5a7c786/users/9c4961b40e044ab9b85be34bb1e97bf6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:42.955232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6148-c5f7-4f44-a5d9-6d88524929c4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:42.955561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6148-c5f7-4f44-a5d9-6d88524929c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:42.956013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6148-c5f7-4f44-a5d9-6d88524929c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.012303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0dfb6148-c5f7-4f44-a5d9-6d88524929c4 None None] MFA Rules not processed for user `9c4961b40e044ab9b85be34bb1e97bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:43.013509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0dfb6148-c5f7-4f44-a5d9-6d88524929c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:43.091189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 748/1502] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:43.104686 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.104973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.105244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.106562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 755/1503] 10.222.0.46 () {54 vars in 964 bytes} [Fri Sep 9 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:21:43.111268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6d3b4-34d4-4e1c-913b-ac38b9b0943b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.112421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6d3b4-34d4-4e1c-913b-ac38b9b0943b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.112882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6d3b4-34d4-4e1c-913b-ac38b9b0943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.144628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:43.188906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3df6d3b4-34d4-4e1c-913b-ac38b9b0943b None None] MFA Rules not processed for user `b670a7afd2264609894dec2feabc4581`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:43.190147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3df6d3b4-34d4-4e1c-913b-ac38b9b0943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:43.264601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 749/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Fri Sep 9 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:21:43.272459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:43.273997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:43.324149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:43.377309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c96046-7082-4087-9173-55b9d02c5957 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:43.450079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:43.450677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.450957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.451229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.452864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:43.553094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:43.554710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:43.558490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:43.573659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 756/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:21:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:21:43.618815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3541-a823-45fd-99c2-a8cedb7ccfb5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.619105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3541-a823-45fd-99c2-a8cedb7ccfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.619385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3541-a823-45fd-99c2-a8cedb7ccfb5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.620685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 750/1506] 10.222.0.46 () {52 vars in 830 bytes} [Fri Sep 9 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:21:43.624174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:43.625401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:43.723291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aba90-b124-4bbf-80d1-4d8c464c8064 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:43.813534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:43.814178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.815012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.815288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] PATH_INFO: `/v3/users/9c4961b40e044ab9b85be34bb1e97bf6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:43.827667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4961b40e044ab9b85be34bb1e97bf6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:43.828702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:43.828702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:21:43.828702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:21:43.829876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:43.839291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:43.894378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4961b40e044ab9b85be34bb1e97bf6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:43.894735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4961b40e044ab9b85be34bb1e97bf6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:43.909104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4961b40e044ab9b85be34bb1e97bf6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:43.931967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4961b40e044ab9b85be34bb1e97bf6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:43.932519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4961b40e044ab9b85be34bb1e97bf6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:43.946719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 757/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:21:43 2022] DELETE /identity/v3/users/9c4961b40e044ab9b85be34bb1e97bf6 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:43.951335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:43.951835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:43.951958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:44.009179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:21:44.010385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:44.086847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:44.100082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 751/1508] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:21:44.110441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:44.112613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:44.217822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdc9d3-3b19-4431-b662-e577fd4a1447 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:44.273824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:44.296126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:44.296740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:44.297019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:44.297292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:44.299033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:44.406242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:44.407751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc463ee-6005-4168-b7ef-c8a55a72260d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:44.426463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 758/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:21:44.753447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:21:44.754701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:21:44.769627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583beef-09d3-44ca-ac57-fb7d97718ee7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:21:44.870676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:21:44.871299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:21:44.871601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:21:44.871868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] PATH_INFO: `/v3/projects/c5b8d0df147f4a8980d3bd48f5a7c786` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:21:44.882141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b8d0df147f4a8980d3bd48f5a7c786)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:21:44.883160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:21:44.883160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:21:44.883160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:21:44.884369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:21:44.943007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b8d0df147f4a8980d3bd48f5a7c786', 'request_id': 'req-f9c4e7fc-dcc5-4046-a669-137ba46338ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:44.954256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b8d0df147f4a8980d3bd48f5a7c786', 'request_id': 'req-f9c4e7fc-dcc5-4046-a669-137ba46338ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:44.968369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b8d0df147f4a8980d3bd48f5a7c786', 'request_id': 'req-f9c4e7fc-dcc5-4046-a669-137ba46338ee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:44.990908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] The token cache is being invalidate because project c5b8d0df147f4a8980d3bd48f5a7c786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:21:44.991198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:44.991812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f9c4e7fc-dcc5-4046-a669-137ba46338ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:21:44.993115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 752/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:21:44 2022] DELETE /identity/v3/projects/c5b8d0df147f4a8980d3bd48f5a7c786 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:21:54.424827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=62 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:21:54.988819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:12.471015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:12.471719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:12.471719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:12.473016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:12.477933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f01f-3005-4846-b0ab-15e3cd128662 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:12.478367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f01f-3005-4846-b0ab-15e3cd128662 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:12.478465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4f01f-3005-4846-b0ab-15e3cd128662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:12.559080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-11a4f01f-3005-4846-b0ab-15e3cd128662 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:12.560321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11a4f01f-3005-4846-b0ab-15e3cd128662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:12.668016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:12.819423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:12.821584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:12.937692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a76ee8-ae18-4543-9bc0-ba1fee9c13f9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:12.943874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:12.944472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:12.944750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:12.945019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:12.952929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:12.954492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794ece2-967c-49e8-9c58-3e4c5d02e44a service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:12.963886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 760/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:14.040071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3454b28-65da-413d-ab25-54fb413c3610 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:14.040646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3454b28-65da-413d-ab25-54fb413c3610 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:14.040646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3454b28-65da-413d-ab25-54fb413c3610 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:14.041956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:14.048027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5f3a-3203-4a96-a450-b22daa4cfa33 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:14.048476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5f3a-3203-4a96-a450-b22daa4cfa33 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:14.048572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5f3a-3203-4a96-a450-b22daa4cfa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:14.068837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8c9d5f3a-3203-4a96-a450-b22daa4cfa33 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:14.070019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c9d5f3a-3203-4a96-a450-b22daa4cfa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:14.077729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:14.225715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3454b28-65da-413d-ab25-54fb413c3610 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:14.240472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3454b28-65da-413d-ab25-54fb413c3610 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:14.245956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:14.246543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:14.246820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:14.247095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:14.256161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:14.257748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b059c2-b09c-448e-9c86-7a7688d1c0b0 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:14.266907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 755/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:14.378407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee98e-4fe3-461e-aecd-ce06efa1d147 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:14.378716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee98e-4fe3-461e-aecd-ce06efa1d147 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:14.379040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddee98e-4fe3-461e-aecd-ce06efa1d147 None None] PATH_INFO: `/v3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:14.380405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 762/1517] 10.222.0.57 () {54 vars in 925 bytes} [Fri Sep 9 21:23:14 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:23:14.386982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82ff2-1dcc-4926-9f83-684c3ff7744a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:14.387269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82ff2-1dcc-4926-9f83-684c3ff7744a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:14.387577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f82ff2-1dcc-4926-9f83-684c3ff7744a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:14.453447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-15f82ff2-1dcc-4926-9f83-684c3ff7744a None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:14.454664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-15f82ff2-1dcc-4926-9f83-684c3ff7744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:14.471485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:14.527635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 756/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Sep 9 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:23.196565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c824f-17b2-400c-9850-77b59ab841c3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:23.197053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c824f-17b2-400c-9850-77b59ab841c3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:23.197158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c824f-17b2-400c-9850-77b59ab841c3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:23.198769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 763/1519] 10.222.0.46 () {54 vars in 961 bytes} [Fri Sep 9 21:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:23:23.203679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e68c-c453-4d0f-8686-df2fdc77c2db None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:23.204141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e68c-c453-4d0f-8686-df2fdc77c2db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:23.204255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e68c-c453-4d0f-8686-df2fdc77c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:23.224154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4c20e68c-c453-4d0f-8686-df2fdc77c2db None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:23.225347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c20e68c-c453-4d0f-8686-df2fdc77c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:23.233757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 757/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Fri Sep 9 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:23.241969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c824f-17b2-400c-9850-77b59ab841c3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:23.256260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c824f-17b2-400c-9850-77b59ab841c3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:23.261142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:23.261730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:23.262009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:23.262280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:23.270781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:23.272389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105e743-b48a-42ac-b267-9e1d36c52746 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:23.281731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 764/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:25.758691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4639f6-9dc9-4cff-b64b-fa965fe8744b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:25.759248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4639f6-9dc9-4cff-b64b-fa965fe8744b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:25.759248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4639f6-9dc9-4cff-b64b-fa965fe8744b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:25.760621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:25.766959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1797cb-1eac-40a0-8010-ffafd8b4d1bd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:25.767371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1797cb-1eac-40a0-8010-ffafd8b4d1bd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:25.767510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1797cb-1eac-40a0-8010-ffafd8b4d1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:25.787812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d1797cb-1eac-40a0-8010-ffafd8b4d1bd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:25.789004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d1797cb-1eac-40a0-8010-ffafd8b4d1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:25.796728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:25.949879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4639f6-9dc9-4cff-b64b-fa965fe8744b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:25.964236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4639f6-9dc9-4cff-b64b-fa965fe8744b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:25.969605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:25.970197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:25.970477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:25.970751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:25.979214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:25.980816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4a597-7aa7-46ea-b318-90e722621343 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:25.991065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 759/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:27.078640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30c7a-589c-4c1d-95fd-16f82b7dbd31 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:27.079200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30c7a-589c-4c1d-95fd-16f82b7dbd31 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:27.079200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30c7a-589c-4c1d-95fd-16f82b7dbd31 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:27.080569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 21:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:27.086774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959e6-f798-4c72-a772-e7c7996284ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:27.087253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959e6-f798-4c72-a772-e7c7996284ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:27.087358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959e6-f798-4c72-a772-e7c7996284ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:27.107577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3f959e6-f798-4c72-a772-e7c7996284ea None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:27.108764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3f959e6-f798-4c72-a772-e7c7996284ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:27.117010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:27.273321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d30c7a-589c-4c1d-95fd-16f82b7dbd31 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:27.288350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d30c7a-589c-4c1d-95fd-16f82b7dbd31 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:27.293792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:27.294378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:27.294656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:27.294927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:27.303646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:27.305221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afc515-ac3d-4fa0-975f-b056796d7c52 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:27.314408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 767/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:31.263400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:31.264027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:31.264027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:31.265350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:31.271313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b804a-1732-4ab7-81ad-5e7f28a1e641 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:31.271750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b804a-1732-4ab7-81ad-5e7f28a1e641 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:31.271902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b804a-1732-4ab7-81ad-5e7f28a1e641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:31.291615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e28b804a-1732-4ab7-81ad-5e7f28a1e641 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:31.292825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e28b804a-1732-4ab7-81ad-5e7f28a1e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:31.300638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:31.600404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:31.602064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:31.682077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e756e-f3a2-4b41-96c4-f87eebbd5db7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:31.685594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:31.686186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:31.686460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:31.686724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:31.696284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:31.697713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57e474-b729-43b3-a259-d9c8bb35b432 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:31.707327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 762/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:33.916372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986abb-e41a-41fb-be0f-f4552713fe5a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:33.917232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986abb-e41a-41fb-be0f-f4552713fe5a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:33.917444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986abb-e41a-41fb-be0f-f4552713fe5a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:33.918934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:33.924317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0b32-fec9-4fb9-8a7a-b7beb5221b9f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:33.924754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0b32-fec9-4fb9-8a7a-b7beb5221b9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:33.924879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e0b32-fec9-4fb9-8a7a-b7beb5221b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:33.944017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f74e0b32-fec9-4fb9-8a7a-b7beb5221b9f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:33.945209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f74e0b32-fec9-4fb9-8a7a-b7beb5221b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:33.952898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:34.237152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986abb-e41a-41fb-be0f-f4552713fe5a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:34.247888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986abb-e41a-41fb-be0f-f4552713fe5a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:34.251422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:34.252039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:34.252347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:34.252619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:34.262031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:34.263464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87babd4-0ca1-4e52-9a3e-167f46bfd2c8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:34.273793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 770/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:37.854146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:37.854728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:37.854728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:37.855995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:37.862387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fbe8-1bab-4d3e-a976-4e894430c99c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:37.862833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fbe8-1bab-4d3e-a976-4e894430c99c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:37.862936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970fbe8-1bab-4d3e-a976-4e894430c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:37.871331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:37.884688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d970fbe8-1bab-4d3e-a976-4e894430c99c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:37.885874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d970fbe8-1bab-4d3e-a976-4e894430c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:37.893611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:38.197056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:38.198756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:38.262941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:38.276614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6d45-28b3-4f8a-87f6-3838dba0ac4b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:38.280161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:38.280769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:38.281054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:38.281324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:38.290539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:38.292000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f652-6edd-4693-a2d2-e472ffc525c1 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:38.301731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 765/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:40.424468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:40.425017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:40.425017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:40.427031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 21:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:23:40.432379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdea1-ee39-4f2a-a926-6b3c5b43c11e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:40.432830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdea1-ee39-4f2a-a926-6b3c5b43c11e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:40.432946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdea1-ee39-4f2a-a926-6b3c5b43c11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:40.452971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-742cdea1-ee39-4f2a-a926-6b3c5b43c11e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:23:40.454206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-742cdea1-ee39-4f2a-a926-6b3c5b43c11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:40.462486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:23:40.766481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:23:40.768185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:23:40.781790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7a911-22a7-4cc1-949e-107cf9b3eda2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:23:40.786417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:23:40.787007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:23:40.787284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:23:40.787573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:23:40.796420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:23:40.797834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35000a-0981-4f45-ac4a-cbc2545d24df service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:23:40.807631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 773/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:23:50.447998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=11 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:23:50.803626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:03.692902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.693851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.693851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.721474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:03.722698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:03.737376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 767/1540] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:03.751010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.752440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.752440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.777258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:03.778492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:03.793540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 774/1541] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:03.799079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:03.812407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d3ded-5a7f-4943-be7a-22cb4fd7879e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:03.820492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:03.821132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.821415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.821689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.822363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:03.823568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:03.824400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:03.846448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 768/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:03.859577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:03.869905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cc46d-6284-4acc-bf32-dd00fbcc389d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:03.878110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:03.879107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:03.879670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:03.880317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:03.881289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:03.883458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:03.943552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 775/1543] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:03 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:03.948261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:03.964264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3b2e3-56e8-42d5-bd82-409032c8b5e2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.040608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.041243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.041527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.041803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.042464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.044662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.080355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 769/1544] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:03 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:04.085183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.105459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796b4b-94ba-4ca3-a0c2-9e64924d1377 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.117507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.118471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.119010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.119491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.120394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.123301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.142362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 776/1545] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:25:04.147007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.164614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1151f-8b88-4522-b784-c3ee3259a972 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.172956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.173606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.173965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.174315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.175048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.176823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cb82-c773-4808-a878-ab441c09604d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.193507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 770/1546] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:04.198640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.212785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce598f-d3f6-40ff-947d-dac37583876f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.220613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.221710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.222248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.222753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] PATH_INFO: `/v3/projects/b604208214494c41a6113396554f4607/users/d0f6b92f6f154c019598438dba703d54/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.246135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b604208214494c41a6113396554f4607, user_id=d0f6b92f6f154c019598438dba703d54, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.249643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d17fc8-16d1-4cd3-afef-2b484589c2ec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.274423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 777/1547] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:04 2022] PUT /identity/v3/projects/b604208214494c41a6113396554f4607/users/d0f6b92f6f154c019598438dba703d54/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:04.280766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a22f-93b2-49d1-8477-20455278a231 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.281099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a22f-93b2-49d1-8477-20455278a231 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.281433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a22f-93b2-49d1-8477-20455278a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.281827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d77b-aa17-4251-af8c-49581df67f2b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.282363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d77b-aa17-4251-af8c-49581df67f2b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.282850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d77b-aa17-4251-af8c-49581df67f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.312836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-75f1d77b-aa17-4251-af8c-49581df67f2b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.314453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75f1d77b-aa17-4251-af8c-49581df67f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.320628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8d32a22f-93b2-49d1-8477-20455278a231 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.321917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d32a22f-93b2-49d1-8477-20455278a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.439176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 778/1548] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:04.440565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8949fa-50b7-4442-bcb6-976d2472b187 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.440854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8949fa-50b7-4442-bcb6-976d2472b187 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.441129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8949fa-50b7-4442-bcb6-976d2472b187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.469465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 771/1549] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.471370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26a2c-a2a0-44a7-b06a-bf7829b4b447 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.472044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26a2c-a2a0-44a7-b06a-bf7829b4b447 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.472556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26a2c-a2a0-44a7-b06a-bf7829b4b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.484249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cc8949fa-50b7-4442-bcb6-976d2472b187 None None] MFA Rules not processed for user `d0f6b92f6f154c019598438dba703d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.485462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc8949fa-50b7-4442-bcb6-976d2472b187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.516178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-10d26a2c-a2a0-44a7-b06a-bf7829b4b447 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.517418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-10d26a2c-a2a0-44a7-b06a-bf7829b4b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.527150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 772/1550] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.528406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad66eb8-6fdb-45c0-b5e6-978d3ae8948f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.528692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad66eb8-6fdb-45c0-b5e6-978d3ae8948f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.528967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad66eb8-6fdb-45c0-b5e6-978d3ae8948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.554054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bad66eb8-6fdb-45c0-b5e6-978d3ae8948f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.555299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bad66eb8-6fdb-45c0-b5e6-978d3ae8948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.565265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 773/1551] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.567195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0921-608e-414d-a358-8cac59042ff5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.568605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0921-608e-414d-a358-8cac59042ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.569301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0921-608e-414d-a358-8cac59042ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.578308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 779/1552] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.580606 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1fcd-6784-4a16-a462-0501afd37e24 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.580897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1fcd-6784-4a16-a462-0501afd37e24 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.581177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1fcd-6784-4a16-a462-0501afd37e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.592371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b18c0921-608e-414d-a358-8cac59042ff5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.593566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b18c0921-608e-414d-a358-8cac59042ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.604005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 774/1553] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.605276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339647-3826-43b7-8a6d-c7e013aeaab4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.605565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339647-3826-43b7-8a6d-c7e013aeaab4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.605884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339647-3826-43b7-8a6d-c7e013aeaab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.606942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8f2d1fcd-6784-4a16-a462-0501afd37e24 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.608381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8f2d1fcd-6784-4a16-a462-0501afd37e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.617289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 780/1554] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.618503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3799-297d-455e-ad2b-83e948b35674 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.618780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3799-297d-455e-ad2b-83e948b35674 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.619049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3799-297d-455e-ad2b-83e948b35674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.627275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a2339647-3826-43b7-8a6d-c7e013aeaab4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.628731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a2339647-3826-43b7-8a6d-c7e013aeaab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.643000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 775/1555] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.643000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158376f-483c-4724-b8f8-2f3382e9741f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.643000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158376f-483c-4724-b8f8-2f3382e9741f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.643000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158376f-483c-4724-b8f8-2f3382e9741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.643386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-68be3799-297d-455e-ad2b-83e948b35674 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.643386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68be3799-297d-455e-ad2b-83e948b35674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.651691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 781/1556] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.652890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.653168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.653437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.663136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:04.668095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4158376f-483c-4724-b8f8-2f3382e9741f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.669340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4158376f-483c-4724-b8f8-2f3382e9741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.674765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.675945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.677316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 776/1557] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.679347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.679671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.679944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.683975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 782/1558] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.684753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.700625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.701840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.710341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 777/1559] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.711127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.721278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e93d45-60a8-4cf3-ad78-328d4b1c54ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.726472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.727072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.727348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.727648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.728314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:04.728611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.730185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.742254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 778/1560] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:04.743063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.744271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18dc40-8040-486f-9d28-3c14da9b028a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.748313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.749358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.749967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.750244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.753347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c63e3-f0d2-471e-a789-823ba7d6e1fd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.758585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.759193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.759437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.759967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.759967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.760423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:04.760720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.760894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda6c37-6a04-463d-94d1-526390d632f8 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.762280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b793c-b7be-4bd0-9eb2-50a982980cf6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.763789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:04.769860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 783/1561] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:04.771826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.772109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.772379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.777518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 779/1562] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:04.779709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.780486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.780677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.799223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.800497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.802599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.803814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.809161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 784/1563] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.810007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.817172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 780/1564] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.818383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.820292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5352-fd58-4372-a49a-4d9bb3e60f77 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.825131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.825730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.826008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.826275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.826947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:04.827213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.828251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:04.828251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:25:04.828251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:25:04.829139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f75e6-5757-44d1-9a63-6434a4751cc8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.829251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3238-fa52-4349-962a-8af577dbae1e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.835426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.836071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.836356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.837165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.837829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:04.838138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.840553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.841757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 785/1565] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:04.842809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.843092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.843367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.860825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 781/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:04.861864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.864475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:04.865850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:04.877928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 786/1567] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:04.879871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.882662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cfe4-3552-4195-9036-9206e066706d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.887929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.889192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.889597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.890143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.892223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.892364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:04.892364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:25:04.892364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:25:04.892891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f2787-7dfc-4054-9805-54947b234286 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:04.893536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.901653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:04.903791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:04.903791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:04.903791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:04.904172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:04.906590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:04.926734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 782/1568] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:04.927838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:04.937981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 787/1569] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:04.938725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.003061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b50d-0a6b-4b38-9347-a670905c46e0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.008025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.008628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.008904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.009167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.010414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:05.010852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.011290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae0277-ab14-4e0e-a216-62aa27273b60 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.012473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.016884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.017502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.017773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.018045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.018690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:05.019453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.021583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.025283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 783/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:05.032804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.034736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 788/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:05.035577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.048314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa701fec-0022-4afb-b5b5-7aed2f5643da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.053236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566b4e-324a-423f-8b15-04d7ddf4ef6f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.058327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.058952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.059261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.059387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.059702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.059992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.060290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.060500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.060606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.061230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.063095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.063386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.099390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 789/1572] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.100041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 784/1573] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.100278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.104408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.131363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.171102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5ea4-d650-4034-ad4b-16f760d797f5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.175863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.176472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.176755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.177238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.178131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:05.178432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.180065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.180906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e998a-9d20-470b-80bc-9793fb2084cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.186305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.187239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.187722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.188149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.188945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.191249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.205703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 790/1574] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:05.206774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.220061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 785/1575] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.221148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.227278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cb134-31c7-466e-8dc9-211e6a1fbfe0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.233636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9ceb1-347f-48c2-b459-c5b020b84ff1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.234605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.235143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.235584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.236331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.237306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.238879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:05.238879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:25:05.238879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:25:05.240825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.240994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.241666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.241958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.242291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.243026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.245332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.263126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.267741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 791/1576] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.267741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.277170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785bdc-0573-4ebb-8e2c-bd197b5e6f11 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.281363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 786/1577] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.282338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.282948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.283237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.283510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.284846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.287419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.291332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.326210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 792/1578] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.327057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.367088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfb75a-61db-4131-8c0c-12683e9a5a14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.420111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7875a-5cb6-47f5-a321-30ab90b7ff8f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.426615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.427936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.428411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.428709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.429467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.433062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.444967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.446044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.446427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.446771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.447501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.449546 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.458828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 793/1579] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 302 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.462306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.472811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f9a9-2755-40a2-b2aa-13b73f665ff9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.477792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 787/1580] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.477792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.483013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.483665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.483948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.484227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.484861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.485067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65adb5b-c99b-49b2-ace8-6c61d38b921a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.487339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.489870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.490474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.490762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.491036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.491710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.493503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.506697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 788/1581] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.506697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.523692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af97f85-78f8-4ddf-a71a-bbfdae31d2d4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.529033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.529643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.529920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.530191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.530834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.534454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 794/1582] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.534454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.534672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.545307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 789/1583] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.546134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.559651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb430a-480c-4565-835a-278990523532 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.585911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.601339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b5b6f-74ba-4c46-b614-ad26d7eb4d65 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.606012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.606626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.606909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.607184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.607848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.609788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.620090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.620701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.620983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.621253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.621876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.624603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.639746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:05.647452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 795/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.649127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.658529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 790/1585] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.658529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.659348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460fb8-cf61-4ebb-b94b-4ff220021179 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.664988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.665588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.665863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.666130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.666768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.667341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351ff0-69a4-44c1-88fe-5e30c68f96bc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.668392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.672448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.673039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.673322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.673608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.674277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.675868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.680238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 796/1586] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.681045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.686579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 791/1587] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.687382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.691236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f2d7-fe29-435f-a263-2a9bf26889fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.698914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.699547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe6555-ce83-4c4a-9c36-fd0c5ce7b04a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.700058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.700396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.700716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.701872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.703950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.705614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.706369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.706653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.706924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.707603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.710451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.717114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 797/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.718118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.728444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338abdb-a708-4a88-be58-45da4544fb6d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.733308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.733904 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.734070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 792/1589] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:05 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:05.735741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.735741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.735933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.736033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.736870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.746828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686840-6a3e-44f6-926d-2cf0ad15f209 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.751148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 798/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.751148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.751411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.752037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.752321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.752590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.753227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.754796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.759092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006070d-20c4-4fdf-a4d7-34ea444332a2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.763742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.764381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.764602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.764877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.765508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.765985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 793/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.766793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.767233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.777008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466bd98-4d48-438f-b460-563e88e708d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.778470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 799/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.779332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.781918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.782651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.782928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.783225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.784154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.785703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.789246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f9e3-fc15-4abc-bf3a-20dd7af3ddc2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.793829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.794414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.794697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.794975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.795629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.795748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 794/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.796471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.800675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.805797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd29a-17a2-4210-92e6-393e4ea01bea admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.810299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.810929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.811207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.814794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 800/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.814936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] PATH_INFO: `/v3/projects/ec1c332dc5c44eee9c7c81787a435533/users/85a24cb16a864a03920ccb359c13ec79/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.815024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.823014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2f99-ce68-4242-8d41-8ea2b2980ffa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.827918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.828114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1c332dc5c44eee9c7c81787a435533, user_id=85a24cb16a864a03920ccb359c13ec79, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.828515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.828796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.829072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.829465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:05.829465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:25:05.829465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:25:05.829739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.831277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.831504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.843553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 801/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.844511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.846588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 795/1596] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:05 2022] PUT /identity/v3/projects/ec1c332dc5c44eee9c7c81787a435533/users/85a24cb16a864a03920ccb359c13ec79/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:05.847268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.860225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5513-bdd1-4f0c-9266-3bb64b9f9abd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.909744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48d6a-651f-4764-a5da-758ea2da3358 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.915167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.915794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.916072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.916341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.916978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.918191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.918850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.919158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.919278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.919446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] PATH_INFO: `/v3/projects/f26a0624f6fd4770baf6347770c28db2/users/2e2411eb77154e7b8fa8f8c7a253ecb9/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.932178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 796/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.932178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.936808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26a0624f6fd4770baf6347770c28db2, user_id=2e2411eb77154e7b8fa8f8c7a253ecb9, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.939391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440a633-469e-42b1-ad2e-27c113f69f6e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:05.939523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.944809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:05.945476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:05.945795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:05.946065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:05.946697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:05.948478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:05.952748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 802/1598] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:05 2022] PUT /identity/v3/projects/f26a0624f6fd4770baf6347770c28db2/users/2e2411eb77154e7b8fa8f8c7a253ecb9/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:05.953947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.962917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 797/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:05.962917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:05.974979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb0a62-7a56-49b7-b9aa-2526a7e6f18f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.008787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.020161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b808a-83e4-4f77-b97c-9dd7eb373882 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.025828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.026482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.026761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.027032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] PATH_INFO: `/v3/projects/c5178215a8034b3f9961687ed3bc45bb/users/95ff3cde924546469ffe31b4e177f754/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.030474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.031132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.031407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.031708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] PATH_INFO: `/v3/projects/777c1bff7c024f91b0eafcad7fce29bc/users/afd147dfc8c648149712518771f7940c/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.042752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5178215a8034b3f9961687ed3bc45bb, user_id=95ff3cde924546469ffe31b4e177f754, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.045290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.045449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.049610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777c1bff7c024f91b0eafcad7fce29bc, user_id=afd147dfc8c648149712518771f7940c, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.052989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264de50-42f1-4b27-8bc9-b123025fc583 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.059231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 803/1600] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:05 2022] PUT /identity/v3/projects/c5178215a8034b3f9961687ed3bc45bb/users/95ff3cde924546469ffe31b4e177f754/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.060339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.068723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 798/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:05 2022] PUT /identity/v3/projects/777c1bff7c024f91b0eafcad7fce29bc/users/afd147dfc8c648149712518771f7940c/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.069954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.070236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.070505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.089991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.091197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.126458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64abc-e612-4a3a-901f-bc60331532fd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.156585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 799/1602] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.157699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.168648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d5bc2-db06-4ca1-bc6b-c44700bdd626 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.173235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.173902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.174180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.174449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] PATH_INFO: `/v3/projects/bc8ed5f414b947e2ab1815f5a775a2c5/users/8c37b14f268f4180a618aa8eeefb5bc6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.188681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.189288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.189568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.189854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8ed5f414b947e2ab1815f5a775a2c5, user_id=8c37b14f268f4180a618aa8eeefb5bc6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.189965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.190576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.192200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.192510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df3a67-5c96-4665-b4f5-063a47af143b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.203321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 804/1603] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:06.204488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.210034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 800/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:06 2022] PUT /identity/v3/projects/bc8ed5f414b947e2ab1815f5a775a2c5/users/8c37b14f268f4180a618aa8eeefb5bc6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.211380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe390-87d3-4936-8f03-582aa5dcb5c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.211706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe390-87d3-4936-8f03-582aa5dcb5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.211983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe390-87d3-4936-8f03-582aa5dcb5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.217667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dbcaa-fb82-4944-89ad-4f81943a00d6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.240166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b0dbe390-87d3-4936-8f03-582aa5dcb5c3 None None] MFA Rules not processed for user `2e2411eb77154e7b8fa8f8c7a253ecb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.241406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b0dbe390-87d3-4936-8f03-582aa5dcb5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.275420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.276377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.276657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.276929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] PATH_INFO: `/v3/projects/61ea6fd33e8943098f064940ba3bb483/users/716b3e70d59d46bdbc73c7287ba4f83c/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.293097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ea6fd33e8943098f064940ba3bb483, user_id=716b3e70d59d46bdbc73c7287ba4f83c, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.296106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c73dd5-3e99-4d39-bf54-c27281b83b95 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.318768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 805/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:06 2022] PUT /identity/v3/projects/61ea6fd33e8943098f064940ba3bb483/users/716b3e70d59d46bdbc73c7287ba4f83c/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.318768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.318768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.318768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.318768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.321449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 801/1606] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.322820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.323306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.323761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.332711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.334023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.343288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.344515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.392156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.395480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 806/1607] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.396436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.397809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.407069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 802/1608] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.407918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.418276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0af7b7-fc6e-498f-b117-818c70328ead None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.422746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.423389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.423698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.423977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] PATH_INFO: `/v3/projects/1ec4f49f69a843d4abd5eb912b518106/users/38636a9bf3e24c5691c08d144385b396/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.438957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec4f49f69a843d4abd5eb912b518106, user_id=38636a9bf3e24c5691c08d144385b396, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.441668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42dfc8-da05-437f-992d-f7ebeaf603a1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.456586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 803/1609] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:06 2022] PUT /identity/v3/projects/1ec4f49f69a843d4abd5eb912b518106/users/38636a9bf3e24c5691c08d144385b396/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:06.467953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1610-2edf-4966-8e49-57b5cc1954c4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.467953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1610-2edf-4966-8e49-57b5cc1954c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.467953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1610-2edf-4966-8e49-57b5cc1954c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.473276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd408f1-e10a-494c-ac33-be488b7b1306 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.490388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7f4b1610-2edf-4966-8e49-57b5cc1954c4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.491848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f4b1610-2edf-4966-8e49-57b5cc1954c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.535741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.536364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.536646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.536916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.553039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 804/1610] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.554256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.554546 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.554814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.585618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] MFA Rules not processed for user `716b3e70d59d46bdbc73c7287ba4f83c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.587050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.603860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.605638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.615125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 807/1611] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:06.615944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.627712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b92bc-be40-4917-a393-893958c3287f service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.639356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.640191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.640480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.640876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.641931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.644135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.651349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.665136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 805/1612] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.665982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.676141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1ed98-7c95-41d8-8c88-a14981fc31f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.684023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 808/1613] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:06.684023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.686070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.686827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.687242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.687486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.704565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.715772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3cc60-66bb-4b35-ae6b-452b20b4f58e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.740950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 806/1614] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:06.743047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.744166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.744599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.772841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70106ffd-b0a0-4236-97eb-769844c95dd5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.778775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.779395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.779712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.779987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.783490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] MFA Rules not processed for user `38636a9bf3e24c5691c08d144385b396`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:06.785226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:06.832101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:06.858771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.860484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.871019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 807/1615] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:06.872158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.872614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 809/1616] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:06.873316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.883014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9cd03-34cf-4dd8-900a-d8d251b16f91 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.884196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5642e-bf69-40c7-9e9e-6fca78855a86 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.886911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.887571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.887833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.888153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.889854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.890592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.890866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.891140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.892026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.893914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57c84-0fce-4b7c-86f3-8efcb28223ec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.899421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.901122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.914093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 808/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:06.915077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.928769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eca999-b96e-4d51-8c07-0bc90ca163fb service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:06.933687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 810/1618] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/projects => generated 355 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:06.934843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:06.935158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c701fb-db77-432c-a923-3b9dd8bfd58b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.935449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c701fb-db77-432c-a923-3b9dd8bfd58b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.935581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.935767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c701fb-db77-432c-a923-3b9dd8bfd58b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.935882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.936123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:06.936814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:06.937894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 811/1619] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 21:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:06.938720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c701fb-db77-432c-a923-3b9dd8bfd58b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:06.939099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcebed-e18e-4379-8062-ef81f12eec26 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:06.971030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 809/1620] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/users => generated 295 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:06.972336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:06.972628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:06.973046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.023926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:07.025682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:07.030458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c701fb-db77-432c-a923-3b9dd8bfd58b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.036654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.037514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.037877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.038188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.039049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.041586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.094351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 812/1621] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:07.094351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.094351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:07.098035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f28f3-09bd-4cbd-8e6d-6700bc7031c6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.111679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.113307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.113806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.114317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.115948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.118535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.136860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 813/1622] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:07.139154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.148191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 810/1623] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:07.149339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.156742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbe0-6cbb-4896-8d0a-b5dcadd797ee admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.163595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.164660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.165239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.165549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.166251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.167077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0924a17-b5ff-4dc8-be67-67b3116131d4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.168014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.175764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.176950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.178459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.178896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] PATH_INFO: `/v3/projects/4b160beffa2e482f9687a53cf7e653eb/users/5e3ae9d97e3541bb8ec74adc85237b5d/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.186335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 814/1624] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:07.186335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.204416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b160beffa2e482f9687a53cf7e653eb, user_id=5e3ae9d97e3541bb8ec74adc85237b5d, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.209438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.211009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c430cb1-45f5-4f27-a5ef-028d652a0dd1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.220781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.221641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.222003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.222364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.232405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 811/1625] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:07 2022] PUT /identity/v3/projects/4b160beffa2e482f9687a53cf7e653eb/users/5e3ae9d97e3541bb8ec74adc85237b5d/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:07.233449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.328582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.330653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.345818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 815/1626] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:07.346893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.361154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e0e-7233-4fe4-9e02-245bc23facb9 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.420663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f16b23-3521-4253-bd5c-5f161b8faa03 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.434902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.437600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.438501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.440903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] PATH_INFO: `/v3/projects/49c4318b000a43ca9c5abe2a755e3e99/users/926c1d103b7d48f18fae1efc43b0e680/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.452398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:07.488297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c4318b000a43ca9c5abe2a755e3e99, user_id=926c1d103b7d48f18fae1efc43b0e680, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.500888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.522756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.523422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 812/1627] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:07 2022] PUT /identity/v3/projects/49c4318b000a43ca9c5abe2a755e3e99/users/926c1d103b7d48f18fae1efc43b0e680/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:07.526924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.527392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.528176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.528314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.531523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.537993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.574255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 816/1628] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:07.576319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.616545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9dd61-29f4-40dc-8c3b-bf84cbaf6464 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.694369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45947e-4bb6-4534-8285-08c05e48882d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.707281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.707987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.708822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.709134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.709928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.711441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.743593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 813/1629] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:07.747499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:07.769623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.771021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.771387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d2cae-fe2a-4951-b280-6aedd1454ac5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:07.771764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.772135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] PATH_INFO: `/v3/projects/4b160beffa2e482f9687a53cf7e653eb/users/5e3ae9d97e3541bb8ec74adc85237b5d/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.780588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b160beffa2e482f9687a53cf7e653eb, user_id=5e3ae9d97e3541bb8ec74adc85237b5d, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.781664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:07.785182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.785626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.787197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] PATH_INFO: `/v3/projects/49c4318b000a43ca9c5abe2a755e3e99/users/926c1d103b7d48f18fae1efc43b0e680/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.789235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c276e-46a2-4f56-9082-cdf59e047e56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.795141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c4318b000a43ca9c5abe2a755e3e99, user_id=926c1d103b7d48f18fae1efc43b0e680, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:07.799921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acef0-2fec-471a-95e3-f2f45f8286b8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:07.831901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 814/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:07 2022] PUT /identity/v3/projects/49c4318b000a43ca9c5abe2a755e3e99/users/926c1d103b7d48f18fae1efc43b0e680/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:07.841050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 817/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:07 2022] PUT /identity/v3/projects/4b160beffa2e482f9687a53cf7e653eb/users/5e3ae9d97e3541bb8ec74adc85237b5d/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:07.842261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fe43-7d27-4a6b-b1b4-741af18a5f6a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.842760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fe43-7d27-4a6b-b1b4-741af18a5f6a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.843199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fe43-7d27-4a6b-b1b4-741af18a5f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.843359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3edb-495a-4a6f-a538-4b9442f28a5c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:07.844500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3edb-495a-4a6f-a538-4b9442f28a5c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:07.845716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c3edb-495a-4a6f-a538-4b9442f28a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:07.889868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:07.902866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ac6c3edb-495a-4a6f-a538-4b9442f28a5c None None] MFA Rules not processed for user `926c1d103b7d48f18fae1efc43b0e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:07.904701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac6c3edb-495a-4a6f-a538-4b9442f28a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:07.910803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-69c6fe43-7d27-4a6b-b1b4-741af18a5f6a None None] MFA Rules not processed for user `5e3ae9d97e3541bb8ec74adc85237b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:07.917634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-69c6fe43-7d27-4a6b-b1b4-741af18a5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:08.064990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 818/1632] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:08.064990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61756-0aac-40d1-af9d-09507fae170d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.064990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61756-0aac-40d1-af9d-09507fae170d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.064990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61756-0aac-40d1-af9d-09507fae170d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.064990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 819/1633] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 21:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:08.075167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43789-83f9-42d6-9ade-d8dda7a7dfb4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.075657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43789-83f9-42d6-9ade-d8dda7a7dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.076077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f43789-83f9-42d6-9ade-d8dda7a7dfb4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.078281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 820/1634] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 21:25:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:08.079169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f43789-83f9-42d6-9ade-d8dda7a7dfb4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:08.119725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 815/1635] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:08.121104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.121406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.121684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.169503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:08.170768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:08.222808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f43789-83f9-42d6-9ade-d8dda7a7dfb4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:08.228641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:08.229786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.230171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.231352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.275692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:08.278575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5683e5-faec-478a-87dd-53ea1780307a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:08.281968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:08.298927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 821/1636] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:08.301013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.301691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.302495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.324733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 816/1637] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:08.329679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:08.343215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:08.347128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:08.358800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ccd5a-3592-424a-9ad2-155f97483040 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:08.375135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:08.378403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.378812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.379583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 822/1638] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:08.379970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.381537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:08.413910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd248-fffc-4121-b45c-82678b3a2327 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:08.420137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:08.421030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.421626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.421754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.441084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:08.444278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9992984-2d3a-4075-a9e9-acbe3254fdbf service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:08.462146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 823/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:08.465883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4183-a647-474e-ac1c-de883fb4cd2b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.466772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4183-a647-474e-ac1c-de883fb4cd2b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.467145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4183-a647-474e-ac1c-de883fb4cd2b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.468822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 824/1640] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 21:25:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:08.475604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.477401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.477925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.519740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:08.529619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:08.568944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:08.570381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:08.575074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 817/1641] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:08.597232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 825/1642] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:08.689365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:08.701677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8eddc9-b837-4670-9faf-5631eadfc2da service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:08.706614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:08.707374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.707783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.708241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.722334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:08.724690 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:08.747149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 818/1643] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:08.838010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:08.854208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae80dd1-0ea2-457c-a795-85e7e1564d1b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:08.858966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:08.859703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:08.860052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:08.860395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:08.973260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:08.975206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9753e6c-8200-48d3-bf70-44438bd9c8a4 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:08.997207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 826/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:09.254983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:09.406384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb2031-260e-4994-aec2-c9b0386650ad service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:09.417983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:09.421717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:09.422325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:09.422757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] PATH_INFO: `/v3/users/38636a9bf3e24c5691c08d144385b396` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:09.424344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38636a9bf3e24c5691c08d144385b396)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:09.428124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:09.428124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Sep 09 21:25:09.428124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: Sep 09 21:25:09.429820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:09.591778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38636a9bf3e24c5691c08d144385b396'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:09.592498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38636a9bf3e24c5691c08d144385b396'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:09.629459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38636a9bf3e24c5691c08d144385b396'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:09.683222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:09.685740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38636a9bf3e24c5691c08d144385b396'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:09.686827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38636a9bf3e24c5691c08d144385b396'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:09.711510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 819/1645] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:09 2022] DELETE /identity/v3/users/38636a9bf3e24c5691c08d144385b396 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:09.716072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883348e-05ad-4f12-8810-bcae927c47c7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:09.716620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883348e-05ad-4f12-8810-bcae927c47c7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:09.717181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883348e-05ad-4f12-8810-bcae927c47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:09.753868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2883348e-05ad-4f12-8810-bcae927c47c7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:09.756484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2883348e-05ad-4f12-8810-bcae927c47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:09.773020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:09.918934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 827/1646] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:10.017877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:10.019975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:10.213243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d420214-9592-469a-9fd4-20c8fbdc83b1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:10.217763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:10.218389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:10.218674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:10.219385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:10.231555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:10.233250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:10.255784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 820/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:10.382521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:10.383282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:10.383716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:10.425897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] MFA Rules not processed for user `95ff3cde924546469ffe31b4e177f754`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:10.428739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:10.641256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:10.643333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:10.651902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 828/1648] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:10.663489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96544ec-c126-4855-ad3a-24f53e9a9cf7 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:10.673564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:10.674982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:10.676409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:10.676805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:10.680131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:10.682376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:10.769628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 821/1649] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:10 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:10.773897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:10.793298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fbfd-43a4-4948-94b9-00ca199fea2a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:10.911921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:10.914274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:10.915930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:10.916542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:10.918993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:10.931051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:11.001296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 829/1650] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:10 2022] POST /identity/v3/users => generated 308 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:11.004377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:11.020211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4484f75-718a-4d83-8b0b-3fa824573a19 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:11.029816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:11.030654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:11.031077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:11.032382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:11.033340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:11.044084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:11.064288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 822/1651] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:11.069056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:11.083989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc2d7-92a6-4f2f-936b-9301e015df17 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:11.098298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:11.101165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:11.101446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:11.101762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] PATH_INFO: `/v3/projects/3682614873c64e8fbb3ed87abe107299/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:11.140033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3682614873c64e8fbb3ed87abe107299, user_id=f7ddb07dcbc74cb38ec4ab8ed14c0e77, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:11.161968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:11.216367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 830/1652] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:11 2022] PUT /identity/v3/projects/3682614873c64e8fbb3ed87abe107299/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:11.222079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:11.251427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62760d-c348-47b9-9d9f-5abe8773f895 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:11.402748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:11.403326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:11.403688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:11.403880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:11.404531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:11.412545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:11.427476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 823/1653] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:11.431924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:11.465865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec11f04-087b-49e8-bfdb-7a3792a34d5f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:11.482632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:11.484729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:11.487187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:11.488970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] PATH_INFO: `/v3/projects/3682614873c64e8fbb3ed87abe107299/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:11.498152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3682614873c64e8fbb3ed87abe107299, user_id=f7ddb07dcbc74cb38ec4ab8ed14c0e77, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:11.514126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9fba7-0a1f-4271-b95a-f9e6a8229daf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:11.518775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:11.566342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 831/1654] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:11 2022] PUT /identity/v3/projects/3682614873c64e8fbb3ed87abe107299/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:11.764809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:11.774242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:11.809146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbca8d-8c9a-4819-a524-017c8d9cd1b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:12.032324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:12.049384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:12.051490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.052559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.058009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] PATH_INFO: `/v3/projects/1ec4f49f69a843d4abd5eb912b518106` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.059644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec4f49f69a843d4abd5eb912b518106)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:12.070475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:12.231881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec4f49f69a843d4abd5eb912b518106', 'request_id': 'req-09447e5f-7e05-4edf-825b-463d4ebd96c7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:12.245779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec4f49f69a843d4abd5eb912b518106', 'request_id': 'req-09447e5f-7e05-4edf-825b-463d4ebd96c7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:12.254748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec4f49f69a843d4abd5eb912b518106', 'request_id': 'req-09447e5f-7e05-4edf-825b-463d4ebd96c7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:12.282218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] The token cache is being invalidate because project 1ec4f49f69a843d4abd5eb912b518106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:12.283313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:12.284663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09447e5f-7e05-4edf-825b-463d4ebd96c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:12.295477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 824/1655] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:11 2022] DELETE /identity/v3/projects/1ec4f49f69a843d4abd5eb912b518106 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:12.297166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.297676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.297761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.344442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:12.345692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:12.465795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 832/1656] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:12.475648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.477521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.477521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.526215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:12.527984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:12.561340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 825/1657] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:12.561457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:12.573682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cb5ff-c045-44f8-9e18-a3158fae32c1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:12.582237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:12.583249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.584162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.584681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.586271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:12.587241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:12.591307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:12.642857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 833/1658] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:12.649147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:12.673084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fb025-a8e8-47c5-ad19-9ad7ece8b6cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:12.679242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:12.680436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.680876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.681292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.682100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:12.685080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:12.767518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 826/1659] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:12 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:12.768640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:12.788665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eec967-6fa8-4f0c-a298-307fa3b29722 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:12.920866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:12.922820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:12.923825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:12.924198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:12.943426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:12.947085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:12.987919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 834/1660] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:12 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:12.989509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:13.026715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71520-d326-4238-a0fc-f7d5679efeb5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:13.040965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:13.041833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.042131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.042395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.043076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:13.044781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:13.068524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 827/1661] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:13.069700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:13.089973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd6d1-80f1-4027-b4ec-b50d37e4c8f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:13.107646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:13.108448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.108793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.109089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.109783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:13.111439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3dc4a-054a-420a-bac7-32da257efd56 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:13.137910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 835/1662] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:13.142240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:13.181983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6916b-2a5a-4fc9-bbea-1e24cfce2338 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:13.203882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:13.204748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.205036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.205321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] PATH_INFO: `/v3/projects/83831d2aa99b41fabf684b3d498d0183/users/304c642b4bf8491fa1d1ef61678428c1/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.267200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83831d2aa99b41fabf684b3d498d0183, user_id=304c642b4bf8491fa1d1ef61678428c1, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:13.273978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:13.317098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 828/1663] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:13 2022] PUT /identity/v3/projects/83831d2aa99b41fabf684b3d498d0183/users/304c642b4bf8491fa1d1ef61678428c1/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:25:13.328454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.329034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.329249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.391781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] MFA Rules not processed for user `304c642b4bf8491fa1d1ef61678428c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:13.393853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:13.543139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:13.579102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 836/1664] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:13.584330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:13.718946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518e195-bf4d-4f06-8b4c-24976df72d28 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:13.730518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:13.731295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.731770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.732161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.732918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:13.734929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:13.800864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 829/1665] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:13 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:13.803140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:13.820005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f389-0fb4-441f-a567-433ab7ddfa7e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:13.950059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:13.950867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:13.951249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:13.952446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:13.953206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:13.956042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.005938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 837/1666] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:13 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:14.009595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:14.024570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088bb8-eebc-4ce0-b7fa-bbe1d19d02b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:14.040609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:14.041197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.042114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.042572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.043505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:14.050463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.052497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:14.076479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 830/1667] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:14.078811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:14.097481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad97b8-a1e2-433d-9e6f-64d884c1e33a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:14.111248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:14.112105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.112578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.112952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] PATH_INFO: `/v3/projects/e4f1a120fd214ed786e66881a028ce07/users/f6bef3d784db464f9b4219b8ea5ec1db/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.151714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f1a120fd214ed786e66881a028ce07, user_id=f6bef3d784db464f9b4219b8ea5ec1db, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:14.154556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.196531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:14.200639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 838/1668] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:14 2022] PUT /identity/v3/projects/e4f1a120fd214ed786e66881a028ce07/users/f6bef3d784db464f9b4219b8ea5ec1db/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:14.220000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd445f-499d-4aba-a582-f916494cbf5f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:14.391611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:14.392359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.392636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.392975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.393638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:14.395446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e6a44-2c6b-4e94-a89b-ea1f7bd01184 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.433662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 831/1669] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:14.434431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:14.446729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba03e-951c-4a55-87ed-d415a5999771 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:14.472814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:14.473522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.478361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.478682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] PATH_INFO: `/v3/projects/e4f1a120fd214ed786e66881a028ce07/users/f6bef3d784db464f9b4219b8ea5ec1db/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.482977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f1a120fd214ed786e66881a028ce07, user_id=f6bef3d784db464f9b4219b8ea5ec1db, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:14.488060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebc11-fef2-4904-bd3a-9f74028e288e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.511775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 839/1670] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:14 2022] PUT /identity/v3/projects/e4f1a120fd214ed786e66881a028ce07/users/f6bef3d784db464f9b4219b8ea5ec1db/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:14.514214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.514654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.515058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.545178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.545644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.546131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.587571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] MFA Rules not processed for user `f6bef3d784db464f9b4219b8ea5ec1db`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:14.593024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:14.602549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] MFA Rules not processed for user `f7ddb07dcbc74cb38ec4ab8ed14c0e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:14.604766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:14.756018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 832/1671] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:14.767691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:14.769199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:14.790850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 840/1672] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:14.846909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7165a7b-2136-49b9-a523-cf1597350208 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:14.851164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:14.851904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:14.852251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:14.852981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:14.863243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:14.864748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce922c-5573-408b-89be-428e88e1e933 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:14.876236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 833/1673] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:15.841912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:15.910647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:16.005983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaebdd-7cbb-48f9-8bdf-30b304bc4c62 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:16.016936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:16.017838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:16.018120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:16.018388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] PATH_INFO: `/v3/users/304c642b4bf8491fa1d1ef61678428c1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:16.024487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=304c642b4bf8491fa1d1ef61678428c1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:16.026609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:16.145332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304c642b4bf8491fa1d1ef61678428c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:16.146626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304c642b4bf8491fa1d1ef61678428c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:16.162170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304c642b4bf8491fa1d1ef61678428c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:16.186738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304c642b4bf8491fa1d1ef61678428c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:16.187481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304c642b4bf8491fa1d1ef61678428c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:16.216592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 841/1674] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:15 2022] DELETE /identity/v3/users/304c642b4bf8491fa1d1ef61678428c1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:16.223216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cf4e0-2314-4a7b-9fcc-5c49040c7d9b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:16.224632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cf4e0-2314-4a7b-9fcc-5c49040c7d9b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:16.225057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cf4e0-2314-4a7b-9fcc-5c49040c7d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:16.272861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-449cf4e0-2314-4a7b-9fcc-5c49040c7d9b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:16.275248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-449cf4e0-2314-4a7b-9fcc-5c49040c7d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:16.331912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:16.423566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 834/1675] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:16.482736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:16.484647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:16.613395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd882-f9a9-4725-8b59-0b33ba0f9ebb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:16.620898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:16.621538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:16.621821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:16.622097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:16.656401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:16.657942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766e8f5-c112-4ba5-8829-3470a8caca5f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:16.682978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 842/1676] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:17.190583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:17.190873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:17.191154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:17.194041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 835/1677] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:17.248814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:17.249380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:17.249380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:17.335154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:17.336438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:17.497021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:17.531873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 843/1678] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:17.557076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:17.559595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:17.590294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8da1f-4b4b-4a2f-9ab0-38cfde77556d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:17.603406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:17.604555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:17.605034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:17.605511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:17.618951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:17.634796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c8c72-1ee0-4e2c-803d-b8b9a17a43db service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:17.665500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:17.667222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:17.686260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 836/1679] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:17.757161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61f08a-96b1-4716-933c-f7337b6326e7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:17.761872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:17.764367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:17.764921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:17.765374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:17.767634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:17.898093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c15e-6424-47d8-8483-fa432427f68f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:17.898385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c15e-6424-47d8-8483-fa432427f68f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:17.898662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221c15e-6424-47d8-8483-fa432427f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:17.965489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:17.969193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:17.984954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4221c15e-6424-47d8-8483-fa432427f68f None None] MFA Rules not processed for user `afd147dfc8c648149712518771f7940c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:17.986542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4221c15e-6424-47d8-8483-fa432427f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.012308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 844/1680] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:18.013279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:18.042452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.082764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83612-fa6a-41db-9cc5-f23351830a75 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:18.098497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:18.099873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:18.099983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:18.100593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] PATH_INFO: `/v3/projects/83831d2aa99b41fabf684b3d498d0183` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:18.108791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83831d2aa99b41fabf684b3d498d0183)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:18.109851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 21:25:18.109851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Sep 09 21:25:18.109851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: Sep 09 21:25:18.111057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:18.226532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 837/1681] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:18.227863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:18.228178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:18.228514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:18.276994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] MFA Rules not processed for user `afd147dfc8c648149712518771f7940c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:18.288141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.351348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83831d2aa99b41fabf684b3d498d0183', 'request_id': 'req-adceeb4c-d712-46a7-a959-64982956ae91'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.370627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83831d2aa99b41fabf684b3d498d0183', 'request_id': 'req-adceeb4c-d712-46a7-a959-64982956ae91'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.394160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83831d2aa99b41fabf684b3d498d0183', 'request_id': 'req-adceeb4c-d712-46a7-a959-64982956ae91'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.443059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 838/1682] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:18.456682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:18.459744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.483938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] The token cache is being invalidate because project 83831d2aa99b41fabf684b3d498d0183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:18.485047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.487148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.501961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 845/1683] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:18 2022] DELETE /identity/v3/projects/83831d2aa99b41fabf684b3d498d0183 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:18.502880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:18.508147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.591006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:18.594337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:18.671083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64532d7-56ea-4360-a120-6b7ff93454f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:18.687747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:18.688536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:18.688825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:18.689101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:18.693729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:18.722902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:18.725110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3e412-a978-402a-bb71-1a8c422b0d11 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:18.762768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adceeb4c-d712-46a7-a959-64982956ae91 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:18.773815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:18.775231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:18.776054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:18.777418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] PATH_INFO: `/v3/users/f6bef3d784db464f9b4219b8ea5ec1db` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:18.778831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 839/1684] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:18.781500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bef3d784db464f9b4219b8ea5ec1db)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:18.784577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:18.947719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bef3d784db464f9b4219b8ea5ec1db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.953898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bef3d784db464f9b4219b8ea5ec1db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:18.972467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bef3d784db464f9b4219b8ea5ec1db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:19.014456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bef3d784db464f9b4219b8ea5ec1db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:19.015862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e53d64d0-0ad9-4c6f-9c62-c3838ce84019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bef3d784db464f9b4219b8ea5ec1db'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:19.061339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 846/1685] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:18 2022] DELETE /identity/v3/users/f6bef3d784db464f9b4219b8ea5ec1db => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:19.421169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:19.421454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:19.422017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:19.424175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 840/1686] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:19.607581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0c6c-8039-419a-91d6-0aa86219f0d7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:19.608078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0c6c-8039-419a-91d6-0aa86219f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:19.608375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd0c6c-8039-419a-91d6-0aa86219f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:19.647274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-36cd0c6c-8039-419a-91d6-0aa86219f0d7 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:19.650704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36cd0c6c-8039-419a-91d6-0aa86219f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:19.798553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:19.800952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:19.843045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 847/1687] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:19.868443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896985-a127-40ed-9bef-2b3cbe641ba6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:19.868730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896985-a127-40ed-9bef-2b3cbe641ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:19.869013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896985-a127-40ed-9bef-2b3cbe641ba6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:19.870372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 848/1688] 10.222.0.46 () {54 vars in 961 bytes} [Fri Sep 9 21:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:25:19.876097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:19.876287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:19.876389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:19.914093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] MFA Rules not processed for user `8ac80d1c221e483989f1e4f5d41a0897`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:19.915582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:19.940942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 849/1689] 10.222.0.46 () {58 vars in 1058 bytes} [Fri Sep 9 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:19.946488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecca30-b650-4564-95bc-7fa3553ecb0b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:19.954517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:19.964337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:19.964997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:19.965335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:19.965656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] PATH_INFO: `/v3/users/95ff3cde924546469ffe31b4e177f754` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:19.967097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ff3cde924546469ffe31b4e177f754)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:19.969191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:19.984843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309965a-688d-4a84-8140-ded88d80b6a6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:19.999510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:20.000186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.000451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.000725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.029539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:20.031083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791e655-a4db-4a37-8c5a-6e84cd7f2eab service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:20.051723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 850/1690] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:20.090036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ff3cde924546469ffe31b4e177f754'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.091575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ff3cde924546469ffe31b4e177f754'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.101312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ff3cde924546469ffe31b4e177f754'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.115749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cf031-c3d8-4509-ab93-3c6cf5497ac4 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.116008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cf031-c3d8-4509-ab93-3c6cf5497ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.116224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cf031-c3d8-4509-ab93-3c6cf5497ac4 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.119027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 851/1691] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 21:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:25:20.123057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.123341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.123774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.135965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ff3cde924546469ffe31b4e177f754'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.139821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ff3cde924546469ffe31b4e177f754'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.182812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 841/1692] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:19 2022] DELETE /identity/v3/users/95ff3cde924546469ffe31b4e177f754 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:20.191670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:20.227079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:20.229165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:20.362892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:20.365559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:20.368372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 852/1693] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:20.450166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:20.478958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a1f5-698d-4dfb-b4e2-d17ebe1d32c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:20.483447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:20.485097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:20.502687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb293159-6e45-49c7-ba44-2ef535ad3065 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:20.514622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:20.515236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.515516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.515809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] PATH_INFO: `/v3/projects/e4f1a120fd214ed786e66881a028ce07` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.519172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f1a120fd214ed786e66881a028ce07)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:20.527281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:20.527685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.527970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.528252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.531473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:20.566541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:20.568220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815182a-6acc-43ca-81d4-4c87a7b3ebb2 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:20.590345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 853/1694] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:20.706127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f1a120fd214ed786e66881a028ce07', 'request_id': 'req-1a7cb649-d010-49db-946c-5ab929f86c11'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.744348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f1a120fd214ed786e66881a028ce07', 'request_id': 'req-1a7cb649-d010-49db-946c-5ab929f86c11'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.754168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f1a120fd214ed786e66881a028ce07', 'request_id': 'req-1a7cb649-d010-49db-946c-5ab929f86c11'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.780286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] The token cache is being invalidate because project e4f1a120fd214ed786e66881a028ce07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:20.781415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.786563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:20.797475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 842/1695] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:20 2022] DELETE /identity/v3/projects/e4f1a120fd214ed786e66881a028ce07 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:20.806670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be47bf7-d485-47d1-a3f1-b066dcf115cb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.807189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be47bf7-d485-47d1-a3f1-b066dcf115cb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.807301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be47bf7-d485-47d1-a3f1-b066dcf115cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.824533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:20.827340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:20.853478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7be47bf7-d485-47d1-a3f1-b066dcf115cb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:20.854718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7be47bf7-d485-47d1-a3f1-b066dcf115cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:20.969971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cb649-d010-49db-946c-5ab929f86c11 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:20.976159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:20.977482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:20.977795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:20.978075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] PATH_INFO: `/v3/projects/c5178215a8034b3f9961687ed3bc45bb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:20.979318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5178215a8034b3f9961687ed3bc45bb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:20.981362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.058063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 854/1696] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:21.065615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.065962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.066190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.082358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5178215a8034b3f9961687ed3bc45bb', 'request_id': 'req-be2c89a0-e63c-48a2-b465-473b1622ca1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.109418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5178215a8034b3f9961687ed3bc45bb', 'request_id': 'req-be2c89a0-e63c-48a2-b465-473b1622ca1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.123800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:21.124273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5178215a8034b3f9961687ed3bc45bb', 'request_id': 'req-be2c89a0-e63c-48a2-b465-473b1622ca1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.125011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:21.142492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] The token cache is being invalidate because project c5178215a8034b3f9961687ed3bc45bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:21.143406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.144288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.149415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 843/1697] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:20 2022] DELETE /identity/v3/projects/c5178215a8034b3f9961687ed3bc45bb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:21.152457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.155033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:21.169174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:21.255820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 855/1698] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:21.260452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.273693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada736d-0d35-4206-b1cf-e074d6d5fe04 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.279252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.280074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.280448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.280813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.281673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:21.282068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.284282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.300764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 856/1699] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:21.305759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c89a0-e63c-48a2-b465-473b1622ca1b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.310703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.320258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.320930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.321217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.321496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] PATH_INFO: `/v3/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.323882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ddb07dcbc74cb38ec4ab8ed14c0e77)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.326275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.329448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ba545-9fb6-48b2-b761-78b311440ffd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.342397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.343180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.344176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.344699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.345454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.347482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.415630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 857/1700] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:21 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:21.417969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.425833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ddb07dcbc74cb38ec4ab8ed14c0e77'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.427171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ddb07dcbc74cb38ec4ab8ed14c0e77'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.431031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:21.444124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ddb07dcbc74cb38ec4ab8ed14c0e77'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.476106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ddb07dcbc74cb38ec4ab8ed14c0e77'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.477039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ddb07dcbc74cb38ec4ab8ed14c0e77'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:21.504467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 844/1701] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:21 2022] DELETE /identity/v3/users/f7ddb07dcbc74cb38ec4ab8ed14c0e77 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:25:21.567015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd714c-be9c-4a22-8acf-8254c38cf710 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.666513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.667652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.668683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.669112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.670627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.673623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.731072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.731774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 858/1702] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:21 2022] POST /identity/v3/users => generated 285 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:21.736700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:21.762805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432722c-3871-4235-8cce-dec830efa209 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.778335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.778971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.779256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.779604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.780220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.781829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.822900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 845/1703] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:21.824237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.837039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abee2c-2658-4e2e-bc73-b469f93dd977 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.850176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.852037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.852326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.853208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.855981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.866430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.886272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.888115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:21.909162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 859/1704] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:21.910273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:21.920828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a964c-ee91-4ed4-b8d3-acc351fc78d1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.927862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.928459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.928742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.929029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] PATH_INFO: `/v3/projects/3682614873c64e8fbb3ed87abe107299` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.930236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3682614873c64e8fbb3ed87abe107299)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.932713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:21.933165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b040-d7fa-4f26-bb89-21de0fff23a1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:21.945296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:21.946224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:21.946662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:21.947091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] PATH_INFO: `/v3/projects/ba6ef93e4d974a9fb1604ad15f48f46b/users/c8ce1e360126492aa9eb7eb61e066cf0/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:21.986168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6ef93e4d974a9fb1604ad15f48f46b, user_id=c8ce1e360126492aa9eb7eb61e066cf0, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:21.989016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ac591-b9af-4f34-ab45-21a00431e58d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.025454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 860/1705] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:21 2022] PUT /identity/v3/projects/ba6ef93e4d974a9fb1604ad15f48f46b/users/c8ce1e360126492aa9eb7eb61e066cf0/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:22.038030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.038674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.039151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.085768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3682614873c64e8fbb3ed87abe107299', 'request_id': 'req-7f3f5777-3473-4f88-a5c2-934785a482d3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:22.101716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3682614873c64e8fbb3ed87abe107299', 'request_id': 'req-7f3f5777-3473-4f88-a5c2-934785a482d3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:22.107249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] MFA Rules not processed for user `c8ce1e360126492aa9eb7eb61e066cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:22.109086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.115754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3682614873c64e8fbb3ed87abe107299', 'request_id': 'req-7f3f5777-3473-4f88-a5c2-934785a482d3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:22.132621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:22.137455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] The token cache is being invalidate because project 3682614873c64e8fbb3ed87abe107299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:22.137759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:22.160059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f3f5777-3473-4f88-a5c2-934785a482d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:22.163568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 846/1706] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:21 2022] DELETE /identity/v3/projects/3682614873c64e8fbb3ed87abe107299 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:22.170223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a02d6-7a32-4a38-a22a-d974b75614d5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.170501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a02d6-7a32-4a38-a22a-d974b75614d5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.170780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a02d6-7a32-4a38-a22a-d974b75614d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.214311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-079a02d6-7a32-4a38-a22a-d974b75614d5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:22.215568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-079a02d6-7a32-4a38-a22a-d974b75614d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.263055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:22.299434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 861/1707] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:22.317971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.317971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.355578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 847/1708] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:22.359014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.359306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.359912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.423508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:22.429827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.451311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5db39-5fea-499e-974c-5a8d059cf808 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.456920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.457637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.458933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.459221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.459982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.462247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.479763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 848/1709] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:22.480942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.496393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751776b-4f69-4def-a49e-cb6110698119 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.514354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.515002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.515288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.515579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.516253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:22.516716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.518475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.524470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 862/1710] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/projects => generated 369 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:22.525545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.538772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.543106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 849/1711] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:22.547671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.564052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a174c-f563-4e7e-a58d-723e3fa21191 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.653420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50617c3a-4643-42e6-9ae1-9e060db1ef00 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.659739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.660378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.660742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.660921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.662978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:22.674988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.677171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.682637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.683358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.684280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.684714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.685877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.687924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.688893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 863/1712] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:22.689436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.712841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765887b-8970-49cc-84a3-421658699a8d service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.742112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 850/1713] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:22.743136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.744074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.744709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.745278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.745580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.746249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.750744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.776866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff6552-861b-4687-83f8-ee7cfd258ac6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.790731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:22.793031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:22.793326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:22.793607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:22.795480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:22.798917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:22.829991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 864/1714] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:22.835598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.838625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 851/1715] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:22.840082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:22.865214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693707f-ea49-45ef-b6a2-e15caa0ed0f6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:22.954218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:23.001717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:23.010995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.011846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.012269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.012641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] PATH_INFO: `/v3/projects/95583402fe6a4516b4f240f00401e9d0/users/8526fc4518b4495c88dba86ff9bf9e0c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.018087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56bb81-cfb2-4d20-b665-b17e3873e636 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.025748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.026386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.026915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.027199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.028021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.029977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.061915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95583402fe6a4516b4f240f00401e9d0, user_id=8526fc4518b4495c88dba86ff9bf9e0c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.066149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.080022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 865/1716] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:22 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:23.081450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.093903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd504f-ee03-4edf-a056-e551746b7603 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.108490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 852/1717] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:22 2022] PUT /identity/v3/projects/95583402fe6a4516b4f240f00401e9d0/users/8526fc4518b4495c88dba86ff9bf9e0c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:23.112206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.199427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.200775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.201230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.201971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.203050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.207470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.227829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 866/1718] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:23.232805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.261825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880b943-979f-4acd-9554-2585e281e79e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.280671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.281819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.282817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.282817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.285531 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.288751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d232e30-ed9f-468f-8c02-6c53fce61042 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.289470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.296257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.297903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.300086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.300821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.303139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.306284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.325734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 853/1719] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:25:23.326874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.330610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 867/1720] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:23.335164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.348833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b55295-b0e2-4474-a898-e5940580dc09 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.354925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.356379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.356776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.357133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] PATH_INFO: `/v3/projects/95583402fe6a4516b4f240f00401e9d0/users/8526fc4518b4495c88dba86ff9bf9e0c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.360841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95583402fe6a4516b4f240f00401e9d0, user_id=8526fc4518b4495c88dba86ff9bf9e0c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.363932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf717e-7709-4b8f-93d1-a3435f06df4d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.369269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17fcb-35a7-4cef-9b0d-5c5ad37394f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.381930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.382726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.383451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.384622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] PATH_INFO: `/v3/projects/aad43e161de4464c8fac32782ef9ca7f/users/5a669bc85ad54ee5a363a53b48364ce2/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.399329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 854/1721] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:23 2022] PUT /identity/v3/projects/95583402fe6a4516b4f240f00401e9d0/users/8526fc4518b4495c88dba86ff9bf9e0c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:23.410387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ec6c-52d9-4e69-b214-ee2d7cd02213 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.410676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ec6c-52d9-4e69-b214-ee2d7cd02213 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.411522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37ec6c-52d9-4e69-b214-ee2d7cd02213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.441733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad43e161de4464c8fac32782ef9ca7f, user_id=5a669bc85ad54ee5a363a53b48364ce2, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.445888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f817e2-10e2-456c-840c-20463231562d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.484388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 868/1722] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:23 2022] PUT /identity/v3/projects/aad43e161de4464c8fac32782ef9ca7f/users/5a669bc85ad54ee5a363a53b48364ce2/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:23.485433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2f37ec6c-52d9-4e69-b214-ee2d7cd02213 None None] MFA Rules not processed for user `8526fc4518b4495c88dba86ff9bf9e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:23.486754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f37ec6c-52d9-4e69-b214-ee2d7cd02213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:23.495051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.495390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.495902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.545094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] MFA Rules not processed for user `5a669bc85ad54ee5a363a53b48364ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:23.546571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:23.666230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 855/1723] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:23.668158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.668460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.668753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.694741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 869/1724] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:23.697432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.699183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:23.712414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] MFA Rules not processed for user `85a24cb16a864a03920ccb359c13ec79`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:23.713640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:23.797903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:23.821804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:23.858953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 856/1725] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:23.859899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.912247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfe2c6-78e3-4f26-a4d8-01df286405f5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:23.925107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:23.926096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:23.926515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:23.927571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:23.954094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:23.956123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:23.984829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 870/1726] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:23.987188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:23.992707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:24.041927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be39d-f626-48bf-b2ce-218e27ec85d1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:24.057567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:24.058205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.058977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.059878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.060540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:24.067950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:24.201606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 857/1727] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:23 2022] POST /identity/v3/projects => generated 381 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:24.204573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:24.229603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7707-b446-41a3-846c-c5d55be1b642 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:24.388678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19c314-9df3-4e10-909d-0546b553b708 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:24.389971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:24.390819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.391224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.392044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.394067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:24.394790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.394948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.395216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.396036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:24.398062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:24.451041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 858/1728] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:24 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:24.455833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:24.481480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681dae2-a6d2-404b-a7af-2f44f5d0aeb0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:24.497771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:24.497771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.498076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.498211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.500285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:24.501980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e91f7-5494-4507-aa7b-7f436f404774 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:24.528425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 859/1729] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:24.530082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.530402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.530859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.569058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:24.575196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:24.593248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] MFA Rules not processed for user `8c37b14f268f4180a618aa8eeefb5bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:24.596361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:24.613217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 871/1730] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:24.637709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:24.660246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:24.672676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ed001-7851-43db-9933-cace98914784 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:24.703835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:24.710406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:24.710406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:24.710406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] PATH_INFO: `/v3/projects/54c557fec9e34b13815f54feca561c06/users/7d3966d74a374e58963880e632b2f90b/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:24.760339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:24.782518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c557fec9e34b13815f54feca561c06, user_id=7d3966d74a374e58963880e632b2f90b, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:24.807014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:24.851894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 872/1731] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:24 2022] PUT /identity/v3/projects/54c557fec9e34b13815f54feca561c06/users/7d3966d74a374e58963880e632b2f90b/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:24.857792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:24.910302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 860/1732] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:25.048003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:25.079049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b19fe-e4c9-49b2-9747-f55a6fd740c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:25.085702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db6be6-793c-4dd8-8e50-0a9f12cb69d3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:25.109882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:25.113627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:25.116862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:25.117788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:25.120295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:25.128481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:25.200282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 873/1733] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:25.203938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:25.259747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c17c-6d21-44eb-8669-ae21072ad392 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:25.278611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:25.280477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:25.281383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:25.281739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] PATH_INFO: `/v3/projects/54c557fec9e34b13815f54feca561c06/users/7d3966d74a374e58963880e632b2f90b/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:25.287404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c557fec9e34b13815f54feca561c06, user_id=7d3966d74a374e58963880e632b2f90b, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:25.296369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414aab-dcb9-4412-a541-c4acbf110b25 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:25.373939 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 874/1734] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:25 2022] PUT /identity/v3/projects/54c557fec9e34b13815f54feca561c06/users/7d3966d74a374e58963880e632b2f90b/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:25.384279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:25.384590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:25.387196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:25.435691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:25.438680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:25.438988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:25.439982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:25.493360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] MFA Rules not processed for user `7d3966d74a374e58963880e632b2f90b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:25.494646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:25.766589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 875/1735] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:25.789354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:25.790974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:25.812592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:25.814619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:25.836588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 861/1736] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 790 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:25.910994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:25.916387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ad90-7f0d-4a41-ba13-cbfd62807974 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:25.924356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:25.926888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:25.926888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:25.926888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:25.965367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:25.965367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f8c6f-0489-4461-89e3-19c141ec83f3 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:25.999010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 876/1737] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:26.133349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:26.211871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54924d-13c2-497c-9232-c0e3920318f5 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:26.217372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:26.218298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:26.218766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:26.219358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:26.298987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:26.388939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:26.390415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:26.421058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 862/1738] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:26.697562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:26.698779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:26.699306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:26.893162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:26.894650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:27.071677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 877/1739] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:27.343913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:27.345528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:27.595868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e12dab-59dc-4b62-8c8a-8036528acba5 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:27.610871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:27.611950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:27.615610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:27.616160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:27.646681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:27.649340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136eb10-82d4-46c8-9e8f-82b6be80ba39 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:27.703773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 863/1740] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:25:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:25:27.748706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:27.757413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:28.072413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d349-e0d2-40b8-810e-ceb7125310b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:28.082515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:28.083943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:28.084226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:28.084502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] PATH_INFO: `/v3/users/d0f6b92f6f154c019598438dba703d54` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:28.089166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f6b92f6f154c019598438dba703d54)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:28.091563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:28.322156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f6b92f6f154c019598438dba703d54'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:28.323208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f6b92f6f154c019598438dba703d54'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:28.362036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f6b92f6f154c019598438dba703d54'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:28.371287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:28.384837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f6b92f6f154c019598438dba703d54'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:28.385961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f6b92f6f154c019598438dba703d54'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:28.423054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 878/1741] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:27 2022] DELETE /identity/v3/users/d0f6b92f6f154c019598438dba703d54 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:28.428569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:28.429060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:28.429408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:28.476924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:28.478860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:28.752855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 864/1742] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:28.786483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:28.793501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:28.959407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd3ee0-d1bd-4d3f-b771-be3fedec04f3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:28.985591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:28.986402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:28.986692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:28.986971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:29.012420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:29.013932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7a0e1-d472-4bf1-8690-4a69f71bf0ca service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:29.039922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 879/1743] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:29.512579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:29.514732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:29.535609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175b41-02f4-4549-8808-a2ce78f92b59 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:29.561810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:29.562808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:29.563269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:29.563751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] PATH_INFO: `/v3/users/c8ce1e360126492aa9eb7eb61e066cf0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:29.571572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ce1e360126492aa9eb7eb61e066cf0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:29.573851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:29.676172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:29.725728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ce1e360126492aa9eb7eb61e066cf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:29.727245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ce1e360126492aa9eb7eb61e066cf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:29.750459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ce1e360126492aa9eb7eb61e066cf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:29.786233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ce1e360126492aa9eb7eb61e066cf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:29.788478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ce1e360126492aa9eb7eb61e066cf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:29.841258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 865/1744] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:29 2022] DELETE /identity/v3/users/c8ce1e360126492aa9eb7eb61e066cf0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:29.841667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474be0f-ab21-4a5a-8294-c772bc43e2ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:29.842215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474be0f-ab21-4a5a-8294-c772bc43e2ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:29.842393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474be0f-ab21-4a5a-8294-c772bc43e2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:29.904704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e474be0f-ab21-4a5a-8294-c772bc43e2ba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:29.914128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e474be0f-ab21-4a5a-8294-c772bc43e2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:30.077601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 880/1745] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:30.094103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:30.095748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:30.264381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce671fcb-2901-4183-890a-a2e25f319b71 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:30.268381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:30.269023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:30.269397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:30.269678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:30.286383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:30.288072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:30.302141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 866/1746] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:30.511413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c13e-d175-45d2-9d41-ae52e903d581 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:30.511413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c13e-d175-45d2-9d41-ae52e903d581 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:30.511744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c13e-d175-45d2-9d41-ae52e903d581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:30.565693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:30.567856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:30.584697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ae96c13e-d175-45d2-9d41-ae52e903d581 None None] MFA Rules not processed for user `8c37b14f268f4180a618aa8eeefb5bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:30.588345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ae96c13e-d175-45d2-9d41-ae52e903d581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:30.607201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb275a-79bd-4c5c-b77b-d42cca12fc18 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:30.633655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:30.634539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:30.634683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:30.635184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] PATH_INFO: `/v3/projects/b604208214494c41a6113396554f4607` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:30.636828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b604208214494c41a6113396554f4607)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:30.648301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:30.707648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 881/1747] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:30.825949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b604208214494c41a6113396554f4607', 'request_id': 'req-b076875f-dc25-406a-82e1-317be09bd6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:30.864348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b604208214494c41a6113396554f4607', 'request_id': 'req-b076875f-dc25-406a-82e1-317be09bd6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:30.886766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b604208214494c41a6113396554f4607', 'request_id': 'req-b076875f-dc25-406a-82e1-317be09bd6ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:30.908709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] The token cache is being invalidate because project b604208214494c41a6113396554f4607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:30.909704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:30.910754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b076875f-dc25-406a-82e1-317be09bd6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:30.922038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 867/1748] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:30 2022] DELETE /identity/v3/projects/b604208214494c41a6113396554f4607 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:30.927292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:30.927862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:30.928106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:30.951337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:30.953289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.013253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:31.140978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 882/1749] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:31.149676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.150336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.150806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.221824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:31.227401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.262708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 868/1750] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:31.267107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.282194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdbe45-1170-46d5-9aa3-e8b2138b4676 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.295952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:31.297569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.298800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.299094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.299781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:31.300084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:31.304748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:31.334677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 883/1751] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:31.339083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.360275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bb304-8d62-40fc-bca4-6a55f344ac1a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.375484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:31.376316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.376539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.376808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.377540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:31.379633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba540c9-57a9-42e5-9c98-64b956c9cd92 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:31.443695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 869/1752] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:31 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:31.448812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.463394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a6a60-f5ec-4d8a-8e3d-5b91ccb0cebf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.528891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf282c6-d137-40ba-86e4-d37c4f788607 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.529178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf282c6-d137-40ba-86e4-d37c4f788607 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.529455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf282c6-d137-40ba-86e4-d37c4f788607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.540151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:31.577600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dbf282c6-d137-40ba-86e4-d37c4f788607 None None] MFA Rules not processed for user `85a24cb16a864a03920ccb359c13ec79`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:31.579792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dbf282c6-d137-40ba-86e4-d37c4f788607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.589151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:31.589778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.590059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.590338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.590993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:31.592985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf028-ae0d-49a5-8158-62ae7e723fa9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:31.618637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 884/1753] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:31 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:31.620105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba209f-9edc-40bb-90c7-63fc966e4482 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.620395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba209f-9edc-40bb-90c7-63fc966e4482 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.620723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba209f-9edc-40bb-90c7-63fc966e4482 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.622530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 885/1754] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 21:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:31.641560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba209f-9edc-40bb-90c7-63fc966e4482 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.662048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 870/1755] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:31.664091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.664091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.664290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.677906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba209f-9edc-40bb-90c7-63fc966e4482 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.685908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:31.686846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.687973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.688490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.689264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:31.690902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:31.714117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 886/1756] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:31.715377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.722334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.726831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:31.728571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.759488 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89002-538c-4ea4-8615-8fc1cc9b8ed9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.765084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:31.765987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:31.766517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:31.766875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] PATH_INFO: `/v3/projects/ba6ef93e4d974a9fb1604ad15f48f46b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:31.768327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6ef93e4d974a9fb1604ad15f48f46b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:31.770334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:31.841868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 871/1757] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:31.842840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.858573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ad6dd-d6dd-4028-9a56-5441f521d315 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:31.878769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6ef93e4d974a9fb1604ad15f48f46b', 'request_id': 'req-cc763fb4-e750-495e-867e-f4d0c9101de7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:31.901955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6ef93e4d974a9fb1604ad15f48f46b', 'request_id': 'req-cc763fb4-e750-495e-867e-f4d0c9101de7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:31.911468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6ef93e4d974a9fb1604ad15f48f46b', 'request_id': 'req-cc763fb4-e750-495e-867e-f4d0c9101de7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:31.928724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] The token cache is being invalidate because project ba6ef93e4d974a9fb1604ad15f48f46b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:31.929444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:31.929841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:31.934435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 887/1758] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:31 2022] DELETE /identity/v3/projects/ba6ef93e4d974a9fb1604ad15f48f46b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:31.935599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:31.937685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:31.984203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:32.013340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.013971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.014252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.014522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.015178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.025225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.052661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 872/1759] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:32.054647 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:32.057029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.072106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef8272-0e9d-4a1a-a442-ee721c01039c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.077783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.078431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.078712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.078982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] PATH_INFO: `/v3/users/8526fc4518b4495c88dba86ff9bf9e0c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.080338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8526fc4518b4495c88dba86ff9bf9e0c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.083586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.084827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc763fb4-e750-495e-867e-f4d0c9101de7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.089981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.090905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.091185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.091463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.093331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.104887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.106736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.118081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 888/1760] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:32.118715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:32.120561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.137563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf8a4d-928b-46a9-9afe-bc2450f10362 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.143503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.144202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.144483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.144760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] PATH_INFO: `/v3/projects/fec5fe8dd1c2442c8e16600d58389f30/users/99be5b94102c40a683c86962db400249/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.154106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8526fc4518b4495c88dba86ff9bf9e0c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:32.154985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8526fc4518b4495c88dba86ff9bf9e0c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:32.156561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:32.167194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8526fc4518b4495c88dba86ff9bf9e0c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:32.178763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec5fe8dd1c2442c8e16600d58389f30, user_id=99be5b94102c40a683c86962db400249, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.188224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b14266-4e82-4ea4-ba03-eed707dc03fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.197451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8526fc4518b4495c88dba86ff9bf9e0c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:32.198410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8526fc4518b4495c88dba86ff9bf9e0c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:32.217479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 889/1761] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:32 2022] PUT /identity/v3/projects/fec5fe8dd1c2442c8e16600d58389f30/users/99be5b94102c40a683c86962db400249/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:32.233477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.233905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.234275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.234582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 873/1762] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:32 2022] DELETE /identity/v3/users/8526fc4518b4495c88dba86ff9bf9e0c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:32.273921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] MFA Rules not processed for user `99be5b94102c40a683c86962db400249`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:32.275211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.398535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 890/1763] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:32.402933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:32.405088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.565036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983103-5ea4-4b92-8db4-3f4d2137e1f6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.587221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.588211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.590511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.590901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.592250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.598648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c2281-1c94-445d-a4a6-471b918218e5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.694455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 874/1764] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:32 2022] POST /identity/v3/projects => generated 371 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:32.695299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:32.719224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a99c0-d304-4802-8370-a20af85691c7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.860720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.861618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.862205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.862807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.864024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.866944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.903388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.903388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.903388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.928634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 891/1765] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:32 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:32.929819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:32.944330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47af6e-3e3d-49c0-adf2-3240abf0721c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:32.951363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:32.952044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:32.952398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:32.952748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:32.953479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:32.955715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:32.965866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:32.967116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:32.971194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 892/1766] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:32.978577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:33.015086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3674a7-b6ce-4df9-bbf0-6112f13e8889 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:33.024100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:33.025521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.026095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.026679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] PATH_INFO: `/v3/projects/d7a8c07f648947a48554d464480ec7b8/users/f2412d4e49304717b549af50c2ca43c6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.031965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:33.070901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a8c07f648947a48554d464480ec7b8, user_id=f2412d4e49304717b549af50c2ca43c6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:33.076032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:33.112288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 893/1767] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:32 2022] PUT /identity/v3/projects/d7a8c07f648947a48554d464480ec7b8/users/f2412d4e49304717b549af50c2ca43c6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:33.117542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:33.126778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 875/1768] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Sep 9 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:33.296997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299fb-25ef-47e3-91b1-9bad9d6c452e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:33.324895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:33.328199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:33.328537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.328884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.330894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.331664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:33.335563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:33.353742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:33.362152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae01efe-3cf2-4702-a39f-9a50a4d2b2ff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:33.413455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 894/1769] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:33.415856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:33.462466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878157-0a3f-4457-8a7a-d856b0f932c6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:33.471210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:33.471938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.472272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.472604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] PATH_INFO: `/v3/projects/d7a8c07f648947a48554d464480ec7b8/users/f2412d4e49304717b549af50c2ca43c6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.475199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a8c07f648947a48554d464480ec7b8, user_id=f2412d4e49304717b549af50c2ca43c6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:33.478020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:33.493033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 895/1770] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:33 2022] PUT /identity/v3/projects/d7a8c07f648947a48554d464480ec7b8/users/f2412d4e49304717b549af50c2ca43c6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:33.494124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:33.496050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:33.639800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:33.641395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.642358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.643668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.688113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:33.696212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7187e-2f94-4d7d-baf4-5b43faa19489 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:33.726339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 876/1771] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:33.728201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.728866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.730124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.757807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:33.759715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270964ba-eea7-45b6-a05a-b01e0704893f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:33.782761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:33.784409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:33.786641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:33.787036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] PATH_INFO: `/v3/projects/95583402fe6a4516b4f240f00401e9d0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:33.790728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] RBAC: Authorizing `identity:delete_project(project_id=95583402fe6a4516b4f240f00401e9d0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:33.799854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:33.835081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] MFA Rules not processed for user `f2412d4e49304717b549af50c2ca43c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:33.838500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:33.956866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95583402fe6a4516b4f240f00401e9d0', 'request_id': 'req-97233098-6ffb-4c37-a57b-953395a3990a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:33.975978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95583402fe6a4516b4f240f00401e9d0', 'request_id': 'req-97233098-6ffb-4c37-a57b-953395a3990a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:33.991508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95583402fe6a4516b4f240f00401e9d0', 'request_id': 'req-97233098-6ffb-4c37-a57b-953395a3990a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:33.998297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:34.036887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] The token cache is being invalidate because project 95583402fe6a4516b4f240f00401e9d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:34.037253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:34.039794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97233098-6ffb-4c37-a57b-953395a3990a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:34.043954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 896/1772] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:33 2022] DELETE /identity/v3/projects/95583402fe6a4516b4f240f00401e9d0 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:34.049791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83898af-d4de-49ef-9591-3ba0f4240f41 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.050868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83898af-d4de-49ef-9591-3ba0f4240f41 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.051316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83898af-d4de-49ef-9591-3ba0f4240f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.126528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c83898af-d4de-49ef-9591-3ba0f4240f41 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:34.130484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c83898af-d4de-49ef-9591-3ba0f4240f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:34.326515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 897/1773] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:34.335596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.338253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.338745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.367829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 877/1774] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 641 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:34.423161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:34.430039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:34.462283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 898/1775] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:34.463593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:34.484965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421c871-232b-424e-a4ab-d2275b6f2a9f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:34.494192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:34.495989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:34.506211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:34.506854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.507138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.507415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.508253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:34.508606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:34.525482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:34.577422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 878/1776] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:34.591928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:34.647422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5f9d3-c17b-48d8-9c38-bbab29bf0884 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:34.672766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:34.673817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.676330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.680201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.680201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:34.683104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:34.751720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05dda-bc7b-417c-bc54-417bd33f7f1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:34.769708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:34.773369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.774430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.774799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.782711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 879/1777] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:34 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:34.783813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:34.951900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1808b-cb59-44bd-a57f-801daa91a096 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:34.976390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:34.977116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:34.977513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:34.977794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:34.978448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:34.980445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.000522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.002802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.048565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 880/1778] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:34 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:35.049558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:35.059132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 899/1779] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:35.111971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a962-58ca-4a0b-af9a-016938c23a35 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:35.139496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:35.140207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.140539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.140779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.141481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.148202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.195849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 881/1780] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:35.197274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:35.226121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2f1c-5ae3-4774-8722-cf2760bf5b3b service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:35.242031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:35.243124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.243489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.243815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.249503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.276489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.278462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:35.281515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:35.307557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 900/1781] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:35.310289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:35.368534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f1b85-b3b6-4825-9990-f1b432cba307 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:35.377454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:35.378414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.378701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.378984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] PATH_INFO: `/v3/projects/82598041f57c44808d4ff4a7ef6052cf/users/c16605f2db6840159b9e9f8f506b7ea9/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.399346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82598041f57c44808d4ff4a7ef6052cf, user_id=c16605f2db6840159b9e9f8f506b7ea9, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.408612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6dcbd5-d895-4036-af84-fa74e9f0bb0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.413177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:35.447714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 901/1782] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:35 2022] PUT /identity/v3/projects/82598041f57c44808d4ff4a7ef6052cf/users/c16605f2db6840159b9e9f8f506b7ea9/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:35.459068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.459475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.459976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.471664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:35.476426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6158b-9214-4ca7-b197-34ad91664995 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:35.501521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] MFA Rules not processed for user `c16605f2db6840159b9e9f8f506b7ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:35.503436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:35.645134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:35.646616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.647048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.647488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.652885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:35.660875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 902/1783] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:35.666611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:35.887415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.893641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:35.922812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 882/1784] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:35.945889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008610ae-be25-4a0d-9e46-5c34d5cd1a20 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:35.969440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:35.970107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:35.970399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:35.970654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:35.977204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:35.979154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.078609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 903/1785] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:35 2022] POST /identity/v3/projects => generated 387 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:36.079955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.103317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36ac0a-1d49-4c75-9c11-f7cbaa11049a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.252124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.252124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.252124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.252124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.253120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.253120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.287847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 883/1786] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:36 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:36.294031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.315701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300db97-6cf0-4efc-aae6-05ae36d3c82d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.322806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.323985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.323985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.324149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.324738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.326418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.339589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 904/1787] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:36.344135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.355142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327cdd-efa1-4d4c-9095-4d741aa06574 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.359891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.360577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.360859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.361134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] PATH_INFO: `/v3/projects/70c431d886dc441eb48859a8a0b58ae5/users/14facabb65524a5b83951ff87a1c0932/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.377090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c431d886dc441eb48859a8a0b58ae5, user_id=14facabb65524a5b83951ff87a1c0932, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.380433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.394534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 884/1788] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:36 2022] PUT /identity/v3/projects/70c431d886dc441eb48859a8a0b58ae5/users/14facabb65524a5b83951ff87a1c0932/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:36.399837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.413680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f755c-da91-472f-a2af-10b2fed00e04 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.475142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.475906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.476278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.476645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.477380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.479073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671aba-f906-4a68-b105-feee87d958d9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.492745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 905/1789] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:36.493906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.504170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80134fb-2b8d-4463-85e1-1f0a6089cd66 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.509429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.510079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.510360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.510759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] PATH_INFO: `/v3/projects/70c431d886dc441eb48859a8a0b58ae5/users/14facabb65524a5b83951ff87a1c0932/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.512689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c431d886dc441eb48859a8a0b58ae5, user_id=14facabb65524a5b83951ff87a1c0932, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.515301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8666cf6-1cff-4e5e-8654-a95a34dd0675 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.517686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:36.529083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 885/1790] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:36 2022] PUT /identity/v3/projects/70c431d886dc441eb48859a8a0b58ae5/users/14facabb65524a5b83951ff87a1c0932/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:36.539770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.540061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.540330 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.549817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:36.574954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] MFA Rules not processed for user `14facabb65524a5b83951ff87a1c0932`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:36.576280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:36.586578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61c44e-416b-4b38-8ab7-432f6c07e978 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.587669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61c44e-416b-4b38-8ab7-432f6c07e978 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.588052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61c44e-416b-4b38-8ab7-432f6c07e978 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.589681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 886/1791] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 21:25:36.608905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.609629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.609912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.644454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:36.646280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:36.657359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 906/1792] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:36.669542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.718657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 887/1793] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:36.725954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61763f-8070-4c1f-90f6-73436ea12ffe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.731057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.731708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.731831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.732212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.732325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.733495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:36.742532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.744231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.748041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced46da-acfb-4e3f-bfb2-49ea593c095e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.753525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.754167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.754476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.754994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.756604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 907/1794] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:36.764431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.765997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa341dc-6724-4756-a860-d9e5754b5960 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.779710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 888/1795] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:25:36.785650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:36.862971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a34452-5e01-4b81-9b59-985cc7ff25e5 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:36.869051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:36.872465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.872768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.873025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] PATH_INFO: `/v3/users/c16605f2db6840159b9e9f8f506b7ea9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.874248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16605f2db6840159b9e9f8f506b7ea9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:36.876277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:36.919206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:36.925902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16605f2db6840159b9e9f8f506b7ea9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:36.927319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16605f2db6840159b9e9f8f506b7ea9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:36.934172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16605f2db6840159b9e9f8f506b7ea9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:36.946730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16605f2db6840159b9e9f8f506b7ea9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:36.947400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16605f2db6840159b9e9f8f506b7ea9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:36.960109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 908/1796] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:36 2022] DELETE /identity/v3/users/c16605f2db6840159b9e9f8f506b7ea9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:36.964712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:36.965028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:36.965307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:36.984013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:36.985206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:37.049567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 889/1797] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:37.059889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:37.061548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:37.138117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a09c93-e2f8-415d-9148-2b305b9d5664 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:37.141907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:37.142516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:37.142796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:37.143100 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:37.153300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:37.154721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:37.165552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 909/1798] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:37.444450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:37.445897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:37.458469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b352-ed70-4c3d-804e-67d6c1607e55 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:37.463202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:37.463892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:37.464169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:37.464437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] PATH_INFO: `/v3/projects/82598041f57c44808d4ff4a7ef6052cf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:37.465547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=82598041f57c44808d4ff4a7ef6052cf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:37.467611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:37.513530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82598041f57c44808d4ff4a7ef6052cf', 'request_id': 'req-c7c3e190-b87a-4a33-9771-723c9a09c2fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.520952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82598041f57c44808d4ff4a7ef6052cf', 'request_id': 'req-c7c3e190-b87a-4a33-9771-723c9a09c2fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.527392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82598041f57c44808d4ff4a7ef6052cf', 'request_id': 'req-c7c3e190-b87a-4a33-9771-723c9a09c2fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.531930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:37.539869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] The token cache is being invalidate because project 82598041f57c44808d4ff4a7ef6052cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:37.540453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.541373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.542874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 890/1799] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:37 2022] DELETE /identity/v3/projects/82598041f57c44808d4ff4a7ef6052cf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:37.546982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:37.557470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a8523-1b57-4a90-a1ed-adb3378eed82 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:37.615168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:37.615850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:37.616139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:37.616416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] PATH_INFO: `/v3/users/14facabb65524a5b83951ff87a1c0932` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:37.617547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14facabb65524a5b83951ff87a1c0932)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:37.619463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:37.666588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14facabb65524a5b83951ff87a1c0932'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.667239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14facabb65524a5b83951ff87a1c0932'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.674055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14facabb65524a5b83951ff87a1c0932'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.675156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:37.687725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14facabb65524a5b83951ff87a1c0932'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.688374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-757808be-896e-45de-a19e-e426bbd7e261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14facabb65524a5b83951ff87a1c0932'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:37.702446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 910/1800] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:37 2022] DELETE /identity/v3/users/14facabb65524a5b83951ff87a1c0932 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:37.987060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:37.988542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:38.002239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e190-b87a-4a33-9771-723c9a09c2fb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.065972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.066602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.066874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.067184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] PATH_INFO: `/v3/projects/70c431d886dc441eb48859a8a0b58ae5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.068370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c431d886dc441eb48859a8a0b58ae5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.070279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.116239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c431d886dc441eb48859a8a0b58ae5', 'request_id': 'req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:38.126638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c431d886dc441eb48859a8a0b58ae5', 'request_id': 'req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:38.139334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c431d886dc441eb48859a8a0b58ae5', 'request_id': 'req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:38.148812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] The token cache is being invalidate because project 70c431d886dc441eb48859a8a0b58ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:38.149107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:38.149908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cfa4ba25-f25e-49c6-bf18-979c2fcb9989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:38.151558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 891/1801] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:37 2022] DELETE /identity/v3/projects/70c431d886dc441eb48859a8a0b58ae5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:38.156416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.156866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.156976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.179020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:38.180333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:38.259823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 911/1802] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:38.268480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.268829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.269171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.291261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:38.292676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:38.302182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 892/1803] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:38.305168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.316277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737daa6-68af-4c8d-b573-235635cbe4e8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.321857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.322529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.322813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.323225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.323918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:38.324219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.325807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.340436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 912/1804] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:38.344777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.355317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d50f6-b255-4437-9bad-918708e18bcd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.364915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.365558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.365833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.366106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.366817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.368719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.399569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 893/1805] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:38.402616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.414337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcb31b-24f4-468e-b133-37598aaa19cd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.471239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.471879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.472164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.472437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.473083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.475818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.497466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 913/1806] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:38.501632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.512493 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c53e92-ce80-4ef2-9f4e-fa89f8b93914 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.517193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.517783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.518058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.518327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.518992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.521462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.531279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 894/1807] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:38.535377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.545128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc4c61-b554-469c-80a2-c634b37156f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.549669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.550254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.550530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.550800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.551594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.553512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98092765-9a1e-460f-a73e-72bc72e56969 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.563760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 914/1808] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:38.568292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.578063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b8aa7-fda5-4c68-ad3d-4e029223b32f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.582573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.584036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.584314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.584585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] PATH_INFO: `/v3/projects/98e601e2997e43cf8f36ea83618b5574/users/223953e8c2394da2b2727ebc7b736cb3/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.599497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e601e2997e43cf8f36ea83618b5574, user_id=223953e8c2394da2b2727ebc7b736cb3, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.602277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.618529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 895/1809] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:38 2022] PUT /identity/v3/projects/98e601e2997e43cf8f36ea83618b5574/users/223953e8c2394da2b2727ebc7b736cb3/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:38.632648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.632990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.633424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.667470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] MFA Rules not processed for user `223953e8c2394da2b2727ebc7b736cb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:38.669391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:38.704492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:38.781114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 915/1810] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:25:38.787421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.818180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:38.867826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de46bf-c6d8-461a-85a6-9e3bd26734cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.872812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:38.873434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:38.873716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:38.873984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:38.874615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:38.876546 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:38.909315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 896/1811] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:38.910577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:38.923797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c29d-4b20-4761-858e-b7fe87126393 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:38.999258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.000209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.000684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.001095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.001834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.004250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.033913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 916/1812] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:38 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:39.041438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.061514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd29f-cd51-452c-8165-34b659cc345e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.066615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.067950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.068456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.068869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.069925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.075754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.094024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 897/1813] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:39.097773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.111416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57dbcc-a803-4a8c-9d39-5e719695da83 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.117153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.117936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.118223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.118505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.119152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.120804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c375d6-f2f9-4906-a27f-b4db57a1acb6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.136066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 917/1814] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:39.136986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.156581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d8e9a-22de-46ab-b9cb-dbea766c7e06 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.164720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.165520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.165800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.174807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] PATH_INFO: `/v3/projects/bc1a70f2a3d8459a888da1167cac7152/users/6835dd17aa7849d9a3705b6e2d620c35/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.196931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1a70f2a3d8459a888da1167cac7152, user_id=6835dd17aa7849d9a3705b6e2d620c35, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.199645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.213541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 898/1815] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:39 2022] PUT /identity/v3/projects/bc1a70f2a3d8459a888da1167cac7152/users/6835dd17aa7849d9a3705b6e2d620c35/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:39.225278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.225844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.225844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.258226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] MFA Rules not processed for user `6835dd17aa7849d9a3705b6e2d620c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:39.259478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:39.352542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 918/1816] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:39.355090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.437780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759febd1-dfa0-41b3-961a-4542b15ce756 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.442610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.443293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.443683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.443961 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.444621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.446521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.483714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 899/1817] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:39 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:39.485081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.498851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28429839-29fa-4eea-a85e-275d0a71d5f5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.542073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:39.584000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.584628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.584910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.585183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.585836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.587797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.616877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 919/1818] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:39 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:25:39.617821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.638562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eefd0-e4d8-4e1f-8702-3a2040bd06db admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.645648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.646346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.646648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.647108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.647848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.649492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.661810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 900/1819] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:39.666318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.678414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14f536-0090-4de5-ad81-08b6bbb4dcb3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.683751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.684502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.684915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.685199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] PATH_INFO: `/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684/users/2690d991435f4e63b9b76a469dea3216/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.701633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ee23fe43c749dfbfaabbefb4ff7684, user_id=2690d991435f4e63b9b76a469dea3216, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.704373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.717914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 920/1820] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:39 2022] PUT /identity/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684/users/2690d991435f4e63b9b76a469dea3216/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:39.721894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.731774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5dc4-2293-4172-b0df-e7e95b23c405 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.776891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:39.794330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.794981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.795222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.795523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.796165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.798186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3679e-cf57-4ba7-8864-bd8b64f66a3c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.810063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 901/1821] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:39.813957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.825074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189263a1-ae9c-468e-872a-027330a4611b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:39.829817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:39.830475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.830806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.831081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] PATH_INFO: `/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684/users/2690d991435f4e63b9b76a469dea3216/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.833077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ee23fe43c749dfbfaabbefb4ff7684, user_id=2690d991435f4e63b9b76a469dea3216, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:39.835755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:39.846139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 921/1822] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:39 2022] PUT /identity/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684/users/2690d991435f4e63b9b76a469dea3216/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:39.856591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:39.856873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:39.857154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:39.886022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] MFA Rules not processed for user `2690d991435f4e63b9b76a469dea3216`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:39.887233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:39.954653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 902/1823] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:39.966135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:39.967995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:40.028587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c86c25-e829-44d1-9a8b-7826413a303e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:40.032302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:40.032909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:40.033190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:40.033464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:40.042933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:40.044388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a134c6a-c337-4198-bdf5-8f406df81801 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:40.054003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 922/1824] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:40.821376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:40.831069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53227061-2daa-4e09-b030-1ba641ece096 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:40.835236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:40.836070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:40.836582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:40.836936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:40.899224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:40.901094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad720912-6178-4bac-b503-de896b7699a6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:40.911162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 903/1825] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:50.049592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=53,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:50.906389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=70 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:54.074151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.075051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.076612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.078704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 923/1826] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:25:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:25:54.085859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.086720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.087289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.118731 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:54.119241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.234150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 904/1827] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:54.254925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:54.256788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.277143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46b1d-3beb-4dcd-8e08-96e8f157f4eb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:54.283656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:54.284538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.284818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.285091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.297328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:54.299465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a850c-ca0f-4c7c-b332-7d0a8d9c2246 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:54.313527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 924/1828] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:54.340145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:54.341620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.444064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc7df-d572-4936-b20e-98921754814b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:54.449405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:54.450111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.450305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.450578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] PATH_INFO: `/v3/users/716b3e70d59d46bdbc73c7287ba4f83c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.451812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=716b3e70d59d46bdbc73c7287ba4f83c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:54.453665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:54.517885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716b3e70d59d46bdbc73c7287ba4f83c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:54.518697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716b3e70d59d46bdbc73c7287ba4f83c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:54.532924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716b3e70d59d46bdbc73c7287ba4f83c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:54.549853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716b3e70d59d46bdbc73c7287ba4f83c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:54.550864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716b3e70d59d46bdbc73c7287ba4f83c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:54.567028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 905/1829] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:54 2022] DELETE /identity/v3/users/716b3e70d59d46bdbc73c7287ba4f83c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:54.571700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.572101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.572470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.595205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:54.596438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.667484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 925/1830] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:54.679937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:54.681449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:54.697373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:54.746819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b975-b732-4496-97cb-6ec8eb0baa21 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:54.751856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:54.752677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:54.752941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:54.753481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:54.764542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:54.766103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:54.783780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 906/1831] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:55.072236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:55.073748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:55.089142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646062f1-145e-4622-ba6d-c1d4af0bb557 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:55.100292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:55.101096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:55.101452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:55.101755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] PATH_INFO: `/v3/projects/61ea6fd33e8943098f064940ba3bb483` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:55.102872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ea6fd33e8943098f064940ba3bb483)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:55.104824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:55.150799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ea6fd33e8943098f064940ba3bb483', 'request_id': 'req-51060a5d-4742-4273-ba93-8cf8ef766990'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.158452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ea6fd33e8943098f064940ba3bb483', 'request_id': 'req-51060a5d-4742-4273-ba93-8cf8ef766990'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.165110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ea6fd33e8943098f064940ba3bb483', 'request_id': 'req-51060a5d-4742-4273-ba93-8cf8ef766990'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.177127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] The token cache is being invalidate because project 61ea6fd33e8943098f064940ba3bb483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:55.177407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.178037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.179647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 926/1832] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:55 2022] DELETE /identity/v3/projects/61ea6fd33e8943098f064940ba3bb483 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:55.183580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:55.193469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16624464-4845-4b10-bc76-e310ae6a9bb0 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:55.251445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:55.252244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:55.252540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:55.252811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] PATH_INFO: `/v3/users/926c1d103b7d48f18fae1efc43b0e680` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:55.253965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] RBAC: Authorizing `identity:delete_user(user_id=926c1d103b7d48f18fae1efc43b0e680)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:55.256782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:55.304465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926c1d103b7d48f18fae1efc43b0e680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.305237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926c1d103b7d48f18fae1efc43b0e680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.312157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926c1d103b7d48f18fae1efc43b0e680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.324537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926c1d103b7d48f18fae1efc43b0e680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.325250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-718b1c09-f26e-45be-a9cf-a7d54e7a405e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926c1d103b7d48f18fae1efc43b0e680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.341805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 907/1833] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:25:55 2022] DELETE /identity/v3/users/926c1d103b7d48f18fae1efc43b0e680 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:55.713498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:55.714943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:55.730485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51060a5d-4742-4273-ba93-8cf8ef766990 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:55.791578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:55.792216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:55.792493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:55.792763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] PATH_INFO: `/v3/projects/49c4318b000a43ca9c5abe2a755e3e99` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:55.793999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c4318b000a43ca9c5abe2a755e3e99)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:55.795915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:55.807782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:55.843734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c4318b000a43ca9c5abe2a755e3e99', 'request_id': 'req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.852057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c4318b000a43ca9c5abe2a755e3e99', 'request_id': 'req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.857867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c4318b000a43ca9c5abe2a755e3e99', 'request_id': 'req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.869512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] The token cache is being invalidate because project 49c4318b000a43ca9c5abe2a755e3e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:25:55.869804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.870497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a9052c05-8cc8-4bb9-ba69-b87f47b07ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:25:55.872248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 927/1834] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:25:55 2022] DELETE /identity/v3/projects/49c4318b000a43ca9c5abe2a755e3e99 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:55.876954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:55.877264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:55.877559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:55.899256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:55.900502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:55.929838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:55.962981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 908/1835] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:55.970824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:55.971115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:55.971392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:55.992009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:55.993221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.006216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 928/1836] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:56.010669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.021453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5561b-1712-4041-8afa-54177ad8660e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.026179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.026948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.027305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.027609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.028358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:25:56.028661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.030231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.045573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 909/1837] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:25:56.049784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.059967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b7058-ddf5-4d72-928a-222d6a07106a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.064628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.065224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.065507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.065778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.066417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.068928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.095381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 929/1838] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:56.100797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.111140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723cddb-a6b0-4e8b-a347-db47e272fbe6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.168180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.168784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.169059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.169329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.169991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.171896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.192680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 910/1839] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:56.196757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.207038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca601d3-d053-4235-af53-21c992adb667 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.212360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.213256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.213756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.214286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.215030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.217139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.231220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 930/1840] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:56.234733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.244927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434de53-a5dc-46f8-89bb-4af4e2b56cfb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.250092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.250697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.250976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.251242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.251906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.253507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f163c1-ae34-4ef3-878f-91fc517aaacf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.264276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 911/1841] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:56.268382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.279062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243d3b3-b07c-4903-b1ec-60446922bb6a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.283585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.284483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.284767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.285042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] PATH_INFO: `/v3/projects/4d8f2e8124eb4671907502b6e53e310c/users/cea350570ea14599bcfe0e0697ad3c6b/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.299595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8f2e8124eb4671907502b6e53e310c, user_id=cea350570ea14599bcfe0e0697ad3c6b, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.302268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.315303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 931/1842] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:56 2022] PUT /identity/v3/projects/4d8f2e8124eb4671907502b6e53e310c/users/cea350570ea14599bcfe0e0697ad3c6b/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:56.325626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.326014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.326291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.354695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] MFA Rules not processed for user `cea350570ea14599bcfe0e0697ad3c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:56.355963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:56.422069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 912/1843] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:56.427253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.492639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e2969-55b9-4c09-88b0-2844dcc5ad5e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.497322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.497845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.498119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.498389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.499020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.500936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.532839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 932/1844] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:56.533979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.545603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf5c8-a995-4790-bac7-d772bceb72b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.572738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:56.604703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.605338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.605622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.605893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.606559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.608490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.646608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 913/1845] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:25:56.648563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.660342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c0c50-9d82-4499-bacb-853e541dd9a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.671035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.671705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.672010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.672261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.672907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.674577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.685704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 933/1846] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:56.689836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.700691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd1a43-52ba-4d41-bb4a-e03656501866 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.705682 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.706330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.706609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.706877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] PATH_INFO: `/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e/users/dbfdd8940ef04a6c8e9cb4083c2e4652/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.723616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f97d7f258a44bbf8e6434b1fa42170e, user_id=dbfdd8940ef04a6c8e9cb4083c2e4652, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.726249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.742845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 914/1847] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:56 2022] PUT /identity/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e/users/dbfdd8940ef04a6c8e9cb4083c2e4652/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:56.746880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.758077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9ac4e-820b-4f3e-be74-082f903fbf9c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.766056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:25:56.818628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.819266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.819573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.819853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.820492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.822089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33450f-a745-4aad-b935-7b08352d793e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.837421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 934/1848] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:25:56.838487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:56.849664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e6a1-8d0c-42f5-b790-c039250f0389 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:56.854929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:56.855636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.855917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.856197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] PATH_INFO: `/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e/users/dbfdd8940ef04a6c8e9cb4083c2e4652/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.858123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f97d7f258a44bbf8e6434b1fa42170e, user_id=dbfdd8940ef04a6c8e9cb4083c2e4652, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:56.861088 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:56.873062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 915/1849] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:25:56 2022] PUT /identity/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e/users/dbfdd8940ef04a6c8e9cb4083c2e4652/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:25:56.884444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:56.884733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:56.885015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:56.930901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] MFA Rules not processed for user `dbfdd8940ef04a6c8e9cb4083c2e4652`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:25:56.932318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:57.023360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 935/1850] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:25:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:25:57.033850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:57.036419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:25:57.096468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d283cc5-3480-4777-8af6-af4b7ee7bb6c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:57.100827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:57.101536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:57.101853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:57.102205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:57.111459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:57.112997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:57.122613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 916/1851] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:25:57.200423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:25:57.210571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1aa4a-0d7a-4c80-b7cc-ce0d2e0386a1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:25:57.214139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:25:57.214731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:25:57.215010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:25:57.215284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:25:57.272037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:25:57.273523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:25:57.282249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 936/1852] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:25:57 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:00.026177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:00.038095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6155c1d-869f-47eb-91bc-07077850e052 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:00.041832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:00.042434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:00.042711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:00.043001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:00.044823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:00.131573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:00.132990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee317145-d392-449a-b3cb-7b9021a922b5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:00.148222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 917/1853] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:00.571111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:00.572671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:00.626068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:00.703219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053db9-ae6a-4202-978c-b5648febbaac service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:00.713019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:00.713630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:00.713863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:00.714569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] PATH_INFO: `/v3/users/5a669bc85ad54ee5a363a53b48364ce2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:00.715931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a669bc85ad54ee5a363a53b48364ce2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:00.718247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:00.819906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a669bc85ad54ee5a363a53b48364ce2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:00.823208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a669bc85ad54ee5a363a53b48364ce2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:00.833866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a669bc85ad54ee5a363a53b48364ce2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:00.862355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a669bc85ad54ee5a363a53b48364ce2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:00.863195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a669bc85ad54ee5a363a53b48364ce2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:00.884780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 937/1854] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:00 2022] DELETE /identity/v3/users/5a669bc85ad54ee5a363a53b48364ce2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:00.889008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:00.889772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:00.890155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:00.917910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:00.919219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:00.952848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:01.033516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 918/1855] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:01.052109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:01.054155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:01.137023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929448c-0a31-4196-8ecf-51f6c6e71c06 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:01.142282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:01.143828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:01.143828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:01.143828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:01.164398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:01.164669 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:01.197087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 938/1856] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:02.132766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:02.134669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:02.150376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac309103-f263-4da4-83e3-ed395a0a130d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:02.155998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:02.156713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:02.156924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:02.157202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] PATH_INFO: `/v3/projects/aad43e161de4464c8fac32782ef9ca7f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:02.158412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad43e161de4464c8fac32782ef9ca7f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:02.161659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:02.210758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad43e161de4464c8fac32782ef9ca7f', 'request_id': 'req-dd705690-0112-4581-a4af-41cabdb65640'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.217859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad43e161de4464c8fac32782ef9ca7f', 'request_id': 'req-dd705690-0112-4581-a4af-41cabdb65640'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.224890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad43e161de4464c8fac32782ef9ca7f', 'request_id': 'req-dd705690-0112-4581-a4af-41cabdb65640'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.234987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] The token cache is being invalidate because project aad43e161de4464c8fac32782ef9ca7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:02.235278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.235932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.237414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 919/1857] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:02 2022] DELETE /identity/v3/projects/aad43e161de4464c8fac32782ef9ca7f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:02.241532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:02.251250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98609e4-5426-4169-9e8b-53041b14ede4 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:02.299743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:02.309769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:02.310397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:02.310677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:02.310950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] PATH_INFO: `/v3/users/7d3966d74a374e58963880e632b2f90b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:02.312107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3966d74a374e58963880e632b2f90b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:02.314027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:02.358407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3966d74a374e58963880e632b2f90b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.359048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3966d74a374e58963880e632b2f90b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.366375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3966d74a374e58963880e632b2f90b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.378467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3966d74a374e58963880e632b2f90b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.379658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ce305ff-0592-4189-92ac-40b2fcfba3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3966d74a374e58963880e632b2f90b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.392779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 939/1858] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:02 2022] DELETE /identity/v3/users/7d3966d74a374e58963880e632b2f90b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:02.691142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:02.693807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:02.713681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705690-0112-4581-a4af-41cabdb65640 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:02.842699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:02.843774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:02.844373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:02.844910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] PATH_INFO: `/v3/projects/54c557fec9e34b13815f54feca561c06` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:02.846540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c557fec9e34b13815f54feca561c06)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:02.849435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:02.935243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:02.936620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c557fec9e34b13815f54feca561c06', 'request_id': 'req-e7836ae1-670f-4eb4-bee9-511ccebffd22'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.951139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c557fec9e34b13815f54feca561c06', 'request_id': 'req-e7836ae1-670f-4eb4-bee9-511ccebffd22'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.960273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c557fec9e34b13815f54feca561c06', 'request_id': 'req-e7836ae1-670f-4eb4-bee9-511ccebffd22'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.971800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] The token cache is being invalidate because project 54c557fec9e34b13815f54feca561c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:02.972029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.973285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7836ae1-670f-4eb4-bee9-511ccebffd22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:02.974930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 920/1859] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:02 2022] DELETE /identity/v3/projects/54c557fec9e34b13815f54feca561c06 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:02.980788 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:02.981075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:02.981391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.001962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:03.003164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.101651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 940/1860] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:03.110185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.110477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.110778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.135701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:03.138028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.150834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 921/1861] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:03.171708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.183985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82a64e-f1f1-4763-9b7c-711537a662ac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.201346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.203890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.205467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.205811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.209890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:03.210229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.211894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.237692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 941/1862] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:03.242154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.251906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bf04a-4c99-4667-bdfa-0ea1fcbcd965 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.257351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.258332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.258629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.259196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.259877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.262117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.304621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 922/1863] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:03.309231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.327911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44957e-f22b-41f1-8abb-b57d3e48fb81 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.412458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.413301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.413747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.414031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.414812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.417231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.427480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:03.447592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 942/1864] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:03.450899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.461651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a72ea-2399-4b6a-ad9a-3c92873fa7c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.468433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.469131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.469481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.469824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.470552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.472389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.484580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 923/1865] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:03.490708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.509258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21657-b202-4a75-8de7-69d2ecd87f28 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.514503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.515326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.515893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.516518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.517440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.519833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.523939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.526268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.533236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 943/1866] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:03.537992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.543987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962c7f-755f-4ae4-8643-041562791d4d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.550166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.550567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.551465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecad51-1969-4749-9424-c2615072fa21 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.551742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.552070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] PATH_INFO: `/v3/users/223953e8c2394da2b2727ebc7b736cb3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.553291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=223953e8c2394da2b2727ebc7b736cb3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.555201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.556922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.558113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.558404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.558684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] PATH_INFO: `/v3/projects/45de9e974a63488ab0add3c4c86bc919/users/04e1cc1b13694f2092737008838855a8/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.578085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45de9e974a63488ab0add3c4c86bc919, user_id=04e1cc1b13694f2092737008838855a8, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.581113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfc766-eaf7-4b32-a89b-6b6c86835bbe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.596746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 944/1867] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:03 2022] PUT /identity/v3/projects/45de9e974a63488ab0add3c4c86bc919/users/04e1cc1b13694f2092737008838855a8/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:03.606963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.607437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.607769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.617552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223953e8c2394da2b2727ebc7b736cb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:03.618267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223953e8c2394da2b2727ebc7b736cb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:03.626609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223953e8c2394da2b2727ebc7b736cb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:03.643061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] MFA Rules not processed for user `04e1cc1b13694f2092737008838855a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:03.644516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.652178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223953e8c2394da2b2727ebc7b736cb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:03.652909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b42f700c-5e0b-466e-a2cd-1048a843bb62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223953e8c2394da2b2727ebc7b736cb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:03.670111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 924/1868] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:03 2022] DELETE /identity/v3/users/223953e8c2394da2b2727ebc7b736cb3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:03.674557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.675002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.675275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.695912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:03.697112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.722362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 945/1869] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:26:03.727063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.728423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.763172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 925/1870] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:26:03.773045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.774481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:03.786836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:03.814747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f19c-448f-4d13-b934-4fedc217498e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.821206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.822041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.822468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.822914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.823735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.825771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.855928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec2df-7ed1-4ff0-bc00-7d1d37c58e83 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.864833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.865213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 946/1871] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:03.865755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.866140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:03.867444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.868013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.881730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.883351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a4c49-7b63-47d5-8e80-7861c52f35fa service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:03.895387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 926/1872] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:03.915313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:03.967885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c2ff-9f3b-4db3-a31a-c37108b5fcb1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:03.974817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:03.975487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:03.976007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:03.976425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:03.977155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:03.979338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a22650-c7bb-4eb8-822e-d3adb6ec294b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.040125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b007-5aee-45e3-b9f1-2bc476f7b05a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.040472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b007-5aee-45e3-b9f1-2bc476f7b05a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.040924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b007-5aee-45e3-b9f1-2bc476f7b05a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.042831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 927/1874] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:26:04.043786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b007-5aee-45e3-b9f1-2bc476f7b05a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.044187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 947/1874] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:03 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:04.065918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.068344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.068672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.085949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b007-5aee-45e3-b9f1-2bc476f7b05a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.093281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.093896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.094183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.094459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.095229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.096898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.116707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:04.118255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.120391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 928/1875] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:04.121440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.123525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.145924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d4fd-ead0-4c94-bd9a-b829f8389fbb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.152708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.154119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.154402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.154681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] PATH_INFO: `/v3/users/cea350570ea14599bcfe0e0697ad3c6b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.156383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea350570ea14599bcfe0e0697ad3c6b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.158893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.234228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 948/1876] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:04.235372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.246401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea350570ea14599bcfe0e0697ad3c6b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.247473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea350570ea14599bcfe0e0697ad3c6b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.263864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea350570ea14599bcfe0e0697ad3c6b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.265391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845bd9-08d3-4fe6-bb92-1ec7343cb12e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.274385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.274931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.275241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.275483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] PATH_INFO: `/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32/users/80f383a49b0441cda4d8dae97f46584c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.281126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea350570ea14599bcfe0e0697ad3c6b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.282089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea350570ea14599bcfe0e0697ad3c6b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.296321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8ece1d5feb49b4b9fc11b5acf91c32, user_id=80f383a49b0441cda4d8dae97f46584c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.299468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.301805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 929/1877] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:04 2022] DELETE /identity/v3/users/cea350570ea14599bcfe0e0697ad3c6b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:04.304347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.306569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.320265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 949/1878] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:04 2022] PUT /identity/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32/users/80f383a49b0441cda4d8dae97f46584c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:04.321194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.323017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.386497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d899dc-f112-4b3c-a9b3-0f2f6481d335 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.407691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:04.418957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9b61-55b3-4aaa-a214-02630159a4c8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.425907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.426846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.427237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.427526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.429321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.441996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.443642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce14ed-2988-4634-95ff-57ec0700cc35 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.455091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 950/1879] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:04.456394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.456675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.457570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.460295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.461168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.461444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.461723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] PATH_INFO: `/v3/users/2e2411eb77154e7b8fa8f8c7a253ecb9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.463041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2411eb77154e7b8fa8f8c7a253ecb9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.465303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.479800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:04.481232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.494093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 951/1880] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:04.494093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.502303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.515007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:04.525367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c775d-ff38-4007-b76e-7634a6afd652 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.532062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.533290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.533615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.534047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.536265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.539397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.556667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2411eb77154e7b8fa8f8c7a253ecb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.557884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2411eb77154e7b8fa8f8c7a253ecb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.563435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 952/1881] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:04.570843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.571481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2411eb77154e7b8fa8f8c7a253ecb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.574183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.592276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2411eb77154e7b8fa8f8c7a253ecb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.593180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2411eb77154e7b8fa8f8c7a253ecb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.615080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 930/1882] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:04 2022] DELETE /identity/v3/users/2e2411eb77154e7b8fa8f8c7a253ecb9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:04.616586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.618007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.694465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94debc5-657a-4dab-b98b-88b83d4c8f3d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.714010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f12fd-de5d-4623-bc14-add0f3ec5b26 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.725341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.726314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.726770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.727207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] PATH_INFO: `/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32/users/80f383a49b0441cda4d8dae97f46584c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.729783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8ece1d5feb49b4b9fc11b5acf91c32, user_id=80f383a49b0441cda4d8dae97f46584c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.733098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.761223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 931/1883] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:04 2022] PUT /identity/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32/users/80f383a49b0441cda4d8dae97f46584c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:04.761962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:04.763044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.763782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.764132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.764471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.766696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.771701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.783514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.785996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380c04-834e-4a74-9418-86a5b42ee720 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.796886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 953/1884] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:04.798282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6b74-47ea-48f0-a06c-fe17b45b132d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.798571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6b74-47ea-48f0-a06c-fe17b45b132d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.798851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6b74-47ea-48f0-a06c-fe17b45b132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.811245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:04.821112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-753b6b74-47ea-48f0-a06c-fe17b45b132d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:04.822377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-753b6b74-47ea-48f0-a06c-fe17b45b132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.875850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c89eb-1edb-479a-ac67-62afb11e0714 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:04.884765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:04.886158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.886638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.886949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] PATH_INFO: `/v3/projects/98e601e2997e43cf8f36ea83618b5574` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.888372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e601e2997e43cf8f36ea83618b5574)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:04.890839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:04.899919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 954/1885] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:04.904686 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:04.904974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:04.905248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:04.938224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] MFA Rules not processed for user `80f383a49b0441cda4d8dae97f46584c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:04.939742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:04.952100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e601e2997e43cf8f36ea83618b5574', 'request_id': 'req-845025d4-eb2c-440f-886a-6015f7c23d3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.965629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e601e2997e43cf8f36ea83618b5574', 'request_id': 'req-845025d4-eb2c-440f-886a-6015f7c23d3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:04.980783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e601e2997e43cf8f36ea83618b5574', 'request_id': 'req-845025d4-eb2c-440f-886a-6015f7c23d3c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.002501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:05.004827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] The token cache is being invalidate because project 98e601e2997e43cf8f36ea83618b5574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:05.005149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.006045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.011671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 932/1886] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:04 2022] DELETE /identity/v3/projects/98e601e2997e43cf8f36ea83618b5574 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:05.013883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.016587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.059365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 955/1887] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:05.060582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.075115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.111439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845025d4-eb2c-440f-886a-6015f7c23d3c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.115461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.116111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.116388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.116656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.118605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.163739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f3735-06b6-47db-a448-25a99c8d96cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.169798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.170623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.170937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.171210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] PATH_INFO: `/v3/users/6835dd17aa7849d9a3705b6e2d620c35` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.172857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6835dd17aa7849d9a3705b6e2d620c35)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.174942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.204595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.206345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.216324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 933/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:05.217297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.219134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.228242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6835dd17aa7849d9a3705b6e2d620c35'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.228922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6835dd17aa7849d9a3705b6e2d620c35'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.234906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:05.238082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6835dd17aa7849d9a3705b6e2d620c35'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.253590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6835dd17aa7849d9a3705b6e2d620c35'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.255156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6835dd17aa7849d9a3705b6e2d620c35'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.274564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 956/1889] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:05 2022] DELETE /identity/v3/users/6835dd17aa7849d9a3705b6e2d620c35 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:05.291984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.294911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.303869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53d836-f128-4a52-bdb7-442770309dbf service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.380062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.381511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.381887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.382371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.386491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.408371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:05.415677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68056-0f84-4374-88e1-39d7dfe45a2f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.424671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.425477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.426710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.427731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] PATH_INFO: `/v3/projects/4d8f2e8124eb4671907502b6e53e310c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.430299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8f2e8124eb4671907502b6e53e310c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.433027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.490689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.492255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.500965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8f2e8124eb4671907502b6e53e310c', 'request_id': 'req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.506324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 934/1890] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:05.512481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8f2e8124eb4671907502b6e53e310c', 'request_id': 'req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.522570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8f2e8124eb4671907502b6e53e310c', 'request_id': 'req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.549685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] The token cache is being invalidate because project 4d8f2e8124eb4671907502b6e53e310c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:05.549985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.551953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.554338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 957/1891] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:05 2022] DELETE /identity/v3/projects/4d8f2e8124eb4671907502b6e53e310c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:05.558508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.584171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0243a-2162-4c19-8039-7b83ea1b8ba7 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.735692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.737233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:05.776430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.777868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.778701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.779313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] PATH_INFO: `/v3/users/dbfdd8940ef04a6c8e9cb4083c2e4652` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.790206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfdd8940ef04a6c8e9cb4083c2e4652)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.796527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.839196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:05.883830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfdd8940ef04a6c8e9cb4083c2e4652'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.885418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfdd8940ef04a6c8e9cb4083c2e4652'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.907508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfdd8940ef04a6c8e9cb4083c2e4652'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.920892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665dd2-3e1c-4c0a-ad54-f218158b7c66 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:05.927943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfdd8940ef04a6c8e9cb4083c2e4652'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.928588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:05.929080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfdd8940ef04a6c8e9cb4083c2e4652'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:05.929408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:05.929697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:05.929977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] PATH_INFO: `/v3/projects/f26a0624f6fd4770baf6347770c28db2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:05.931298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26a0624f6fd4770baf6347770c28db2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:05.933789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:05.950259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 935/1892] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:05 2022] DELETE /identity/v3/users/dbfdd8940ef04a6c8e9cb4083c2e4652 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:05.951243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:05.954817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.048405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26a0624f6fd4770baf6347770c28db2', 'request_id': 'req-9cb1da05-ebb6-4696-9c92-e7bb084da84a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.060472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26a0624f6fd4770baf6347770c28db2', 'request_id': 'req-9cb1da05-ebb6-4696-9c92-e7bb084da84a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.094545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26a0624f6fd4770baf6347770c28db2', 'request_id': 'req-9cb1da05-ebb6-4696-9c92-e7bb084da84a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.112513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8bac7-ddad-495e-adfd-ee2e9e5d8dee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.127131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] The token cache is being invalidate because project f26a0624f6fd4770baf6347770c28db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:06.127431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.129893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.140804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 958/1893] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:05 2022] DELETE /identity/v3/projects/f26a0624f6fd4770baf6347770c28db2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:06.142845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.145894 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.188465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:06.245337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.246106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.246506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.247160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] PATH_INFO: `/v3/projects/bc1a70f2a3d8459a888da1167cac7152` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.248213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1a70f2a3d8459a888da1167cac7152)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.251686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.267811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1da05-ebb6-4696-9c92-e7bb084da84a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.273847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80f383a49b0441cda4d8dae97f46584c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.274587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.274882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.275159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.275850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:06.276155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.278475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.293260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 959/1894] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:06.294099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.295810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.322754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73ff7-c42e-411c-ac48-7104bb95a447 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.338400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:06.341448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1a70f2a3d8459a888da1167cac7152', 'request_id': 'req-d2edfee3-ff7c-40b2-9932-b244bb079af5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.361116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1a70f2a3d8459a888da1167cac7152', 'request_id': 'req-d2edfee3-ff7c-40b2-9932-b244bb079af5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.371174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1a70f2a3d8459a888da1167cac7152', 'request_id': 'req-d2edfee3-ff7c-40b2-9932-b244bb079af5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.403416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] The token cache is being invalidate because project bc1a70f2a3d8459a888da1167cac7152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:06.403770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.404424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.408913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 936/1895] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:05 2022] DELETE /identity/v3/projects/bc1a70f2a3d8459a888da1167cac7152 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:06.408913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.410839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.418720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.419425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.419801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.420074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] PATH_INFO: `/v3/users/5e3ae9d97e3541bb8ec74adc85237b5d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.421711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3ae9d97e3541bb8ec74adc85237b5d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.423679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.499443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3ae9d97e3541bb8ec74adc85237b5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.504503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3ae9d97e3541bb8ec74adc85237b5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.515770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3ae9d97e3541bb8ec74adc85237b5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.527162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edfee3-ff7c-40b2-9932-b244bb079af5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.532195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3ae9d97e3541bb8ec74adc85237b5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.533019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3ae9d97e3541bb8ec74adc85237b5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.533271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80f383a49b0441cda4d8dae97f46584c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.533883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.534439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.534707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.535389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.537366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.549767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 960/1896] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:06 2022] DELETE /identity/v3/users/5e3ae9d97e3541bb8ec74adc85237b5d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:06.550770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.552367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.587276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 937/1897] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:06 2022] POST /identity/v3/projects => generated 355 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:06.587460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.590974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.645708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:06.682245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13a38c-6969-4f53-b34e-6a465cd83fc3 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.693428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.694132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.694425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.695545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] PATH_INFO: `/v3/users/2690d991435f4e63b9b76a469dea3216` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.698500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2690d991435f4e63b9b76a469dea3216)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.699473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e080a88-0cf8-43e5-9847-4d7a8ef39244 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:06.703582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.705127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:06.706019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.706500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.707739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] PATH_INFO: `/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.709375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f97d7f258a44bbf8e6434b1fa42170e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:06.711497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:06.786600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f97d7f258a44bbf8e6434b1fa42170e', 'request_id': 'req-1a12c669-2854-472f-9682-3d63eac3ea22'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.804309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f97d7f258a44bbf8e6434b1fa42170e', 'request_id': 'req-1a12c669-2854-472f-9682-3d63eac3ea22'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.813698 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f97d7f258a44bbf8e6434b1fa42170e', 'request_id': 'req-1a12c669-2854-472f-9682-3d63eac3ea22'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.837463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2690d991435f4e63b9b76a469dea3216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.838285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2690d991435f4e63b9b76a469dea3216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.839994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] The token cache is being invalidate because project 3f97d7f258a44bbf8e6434b1fa42170e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:06.840240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.840923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.848834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 961/1898] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:06 2022] DELETE /identity/v3/projects/3f97d7f258a44bbf8e6434b1fa42170e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:06.850718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:06.856587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.860448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2690d991435f4e63b9b76a469dea3216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.871967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:06.892557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2690d991435f4e63b9b76a469dea3216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.895523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22103e72-6f5a-4c24-ac95-4e752b111f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2690d991435f4e63b9b76a469dea3216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:06.961283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 938/1899] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:06 2022] DELETE /identity/v3/users/2690d991435f4e63b9b76a469dea3216 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:06.963017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:06.964087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:06.964587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:06.993412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:06.995091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:06.998238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12c669-2854-472f-9682-3d63eac3ea22 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.114906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80f383a49b0441cda4d8dae97f46584c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.115639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.116012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.116358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.117318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:07.117406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.119741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a85a7-784e-4e38-95d6-2df0e8cd9336 tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.142922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 962/1900] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:07.153110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 939/1901] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:07.161131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.161508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.161794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.173432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:07.199677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:07.199677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.201022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.217366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9cd06-b593-4431-9c2d-dc1b61cbb1e5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.223020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 963/1902] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:07.224134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.228397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80f383a49b0441cda4d8dae97f46584c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.229418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.230763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.231243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] PATH_INFO: `/v3/projects/c3a1d87ca4a549a7b435b27305ab483e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.233083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c3a1d87ca4a549a7b435b27305ab483e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.235599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.242834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd8223-72f5-4880-99ea-6d3227713c13 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.252306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.253375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.254552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.255226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.256196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:07.258813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.261062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.287643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 964/1903] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:07.287643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.287643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.311432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bb215-b5a8-4883-8191-9e33d658f316 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.322108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.322753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.323041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.324104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] PATH_INFO: `/v3/projects/4b160beffa2e482f9687a53cf7e653eb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.325245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b160beffa2e482f9687a53cf7e653eb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.329625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.335849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a1d87ca4a549a7b435b27305ab483e', 'request_id': 'req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.353090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a1d87ca4a549a7b435b27305ab483e', 'request_id': 'req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.367584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a1d87ca4a549a7b435b27305ab483e', 'request_id': 'req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.386832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] The token cache is being invalidate because project c3a1d87ca4a549a7b435b27305ab483e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:07.387597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.388799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.390575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 940/1904] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:07 2022] DELETE /identity/v3/projects/c3a1d87ca4a549a7b435b27305ab483e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:07.391611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.392969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.408203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b160beffa2e482f9687a53cf7e653eb', 'request_id': 'req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.430211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b160beffa2e482f9687a53cf7e653eb', 'request_id': 'req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.441060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b160beffa2e482f9687a53cf7e653eb', 'request_id': 'req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.483487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] The token cache is being invalidate because project 4b160beffa2e482f9687a53cf7e653eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:07.483754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.485371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e654b4ae-7f30-4e2e-acb2-7ad945e395bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.486277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a9d2c-cf98-4ea6-8254-2d6fa40cfd4e tempest-VolumeQuotaClassesTest-981937997 tempest-VolumeQuotaClassesTest-981937997-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.488045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 965/1905] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:07 2022] DELETE /identity/v3/projects/4b160beffa2e482f9687a53cf7e653eb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:07.495175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.495549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.495826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.515185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:07.520688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:07.523313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.622218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.622977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.623263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.623678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.624439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.626680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eefe45-56a5-44f0-af6c-94c8d7288c8e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.634423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 966/1906] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:07.635368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.636918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.655671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b497ef7-516f-4533-a04d-035c18c0200c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.661697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.662341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.662620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.662897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] PATH_INFO: `/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.664863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ee23fe43c749dfbfaabbefb4ff7684)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.666309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.667942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 941/1907] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/projects => generated 371 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:07.670414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.670706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.671321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.703201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:07.704703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.727104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ee23fe43c749dfbfaabbefb4ff7684', 'request_id': 'req-67b4175a-57a6-457d-baf2-8406646294f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.739176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ee23fe43c749dfbfaabbefb4ff7684', 'request_id': 'req-67b4175a-57a6-457d-baf2-8406646294f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.748052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:07.750335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ee23fe43c749dfbfaabbefb4ff7684', 'request_id': 'req-67b4175a-57a6-457d-baf2-8406646294f2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.767386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] The token cache is being invalidate because project 83ee23fe43c749dfbfaabbefb4ff7684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:07.768807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.771165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:07.779076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 967/1908] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:07 2022] DELETE /identity/v3/projects/83ee23fe43c749dfbfaabbefb4ff7684 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:07.779076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.779892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.816103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 942/1909] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:07.817590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:07.818910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.837620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245efbc-6e14-4d4a-a46c-8b483dfa2b47 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.879450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4175a-57a6-457d-baf2-8406646294f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:07.885756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.886790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.887279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.887853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.888746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.891191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a26479-6c22-4f70-8199-826b91620aec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.923931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 968/1910] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:07.929102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.929401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.929731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.930219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:07.931198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:07.931573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:07.932000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] PATH_INFO: `/v3/users/04e1cc1b13694f2092737008838855a8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:07.934901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e1cc1b13694f2092737008838855a8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:07.937291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:07.963927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:07.968591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:07.998094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 969/1911] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:07.999610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.021263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cedd7-9db5-45ec-b879-7004aac61b34 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.021878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:08.026517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.027258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.027646 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.028664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.028988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:08.029286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.030888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.038220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e1cc1b13694f2092737008838855a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:08.040942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e1cc1b13694f2092737008838855a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:08.056516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e1cc1b13694f2092737008838855a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:08.068282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 970/1912] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:08.069279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.075001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.089114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e1cc1b13694f2092737008838855a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:08.090620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-66c2fa46-6744-4247-8a80-316681c40fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e1cc1b13694f2092737008838855a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:08.105868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745dfae-821e-4084-b671-1a2a5f869479 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.110288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 943/1913] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:07 2022] DELETE /identity/v3/users/04e1cc1b13694f2092737008838855a8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:08.115575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.115721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.116036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.148976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:08.161296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.246418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.247065 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.247358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.247638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.248452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.251067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971372c0-08b6-4a6e-acef-15c55e3294b2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.258456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 944/1914] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:08.259358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.262015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.278095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5693c-de0d-4a44-894b-1f88a6296a09 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.278800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 971/1915] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:08.280419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.280707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.281099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.288669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.289392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.289863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.290144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.290793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.292948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.322497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:08.326604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.353987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 945/1916] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/projects => generated 409 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:08.353987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.354612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 972/1917] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:08.354612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.368564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7b35-4206-4e47-a3e7-52da023bc5a2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.425833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:08.454145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.454841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.455039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.455302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.456172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.458585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.465689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413ea1-2d9c-474a-a88d-b3c7cb41f315 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.471747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.473211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.474338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.474622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.475377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:08.475706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.477592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.480769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 973/1918] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:08.481968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.492714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2e843-cc5a-405e-83bd-499ccfc09e3a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.497638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 946/1919] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:08.498495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.500897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.501271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.501631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.502420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.502533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.505095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.505463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.529715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 974/1920] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/users => generated 305 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:08.530349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.541598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662788e-e8e8-4a26-98e1-7ef897f1f16b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.546191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:08.546766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.547571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.547772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.548093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] PATH_INFO: `/v3/projects/af0a3b65874744918c60a2cf098cdbac/users/b1c18aadf10d46018ee874f47fb9b9d6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.567195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0a3b65874744918c60a2cf098cdbac, user_id=b1c18aadf10d46018ee874f47fb9b9d6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.570053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.581511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4bf5-2736-4266-8a03-498211e84e27 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.588628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 975/1921] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:08 2022] PUT /identity/v3/projects/af0a3b65874744918c60a2cf098cdbac/users/b1c18aadf10d46018ee874f47fb9b9d6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:08.589054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.643548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.645041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.645410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.645754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.678820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e9bc-a40a-4296-9c71-095d1fdf9d89 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.684447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.685367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.685722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.686198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.686919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.689142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.723878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 976/1922] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:08.728452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.729851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.732059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c00ca6-1022-4def-b9bf-089783318a9e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.746221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 947/1923] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:08.748433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.748938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.749079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.787421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:08.789410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.818922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cecf3-0b5a-43f5-9860-2fac7c0c3863 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.824267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.825282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.825556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.825829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.827099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.828586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.845063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 977/1924] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:08.845063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.857661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53083a-9983-40d5-848c-1faea9bad0fb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.862897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.863631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.863912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.864383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.865205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 948/1925] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:08.865325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.867120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.869235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.881060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f813f-f987-4cfd-8636-6f5b6457fbe5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.886956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.888347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.888643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.888957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.889795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:08.891890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:08.915408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 949/1926] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:08.915408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.915408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:08.940568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 978/1927] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:08 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:08.944141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:08.960266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:08.975810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9174ff8-90f6-40c5-91c6-5259656fa790 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:08.981903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:08.983248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:08.983248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:08.984029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] PATH_INFO: `/v3/projects/bdfe4124a70c4da78a6ee17b150d5523/users/b7a8294318064eb6929055e52b71309d/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:08.986599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:09.005266 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfe4124a70c4da78a6ee17b150d5523, user_id=b7a8294318064eb6929055e52b71309d, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.008026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.014975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8811b-cd49-4aee-8a13-89f969a8256e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.019176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.020583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.021113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.021289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.029058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 979/1928] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:08 2022] PUT /identity/v3/projects/bdfe4124a70c4da78a6ee17b150d5523/users/b7a8294318064eb6929055e52b71309d/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:09.029890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.044405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.045753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480ae43-20ba-4af4-bb67-a256ff188bf3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.057518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 950/1929] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:09.058874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.059281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.059574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.097235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] MFA Rules not processed for user `b7a8294318064eb6929055e52b71309d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:09.099357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:09.125599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f82e0-58b6-419d-8f57-88080ed5dcf2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.133169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.133775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.134054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.134327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.134982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.137292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.155027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 980/1930] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:09.158015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.176918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8dd26-cd54-41c9-817d-db39470de0a3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.188878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.190916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.192796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.193095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.193831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.198593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.214453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 951/1931] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:09.215290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.219355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:09.221315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 981/1932] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:26:09.222317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.238583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda42900-d8bd-4e15-a468-b3665d5c8d68 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.244733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.245481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.245763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.246077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.246667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.247783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72626-6f8b-4007-8e8c-4bc5b557457e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.251817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.282443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.283093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.283364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.286215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] PATH_INFO: `/v3/projects/45de9e974a63488ab0add3c4c86bc919` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.287481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45de9e974a63488ab0add3c4c86bc919)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.293217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.296663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 982/1933] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:09 2022] POST /identity/v3/projects => generated 409 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:09.299347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.380982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45de9e974a63488ab0add3c4c86bc919', 'request_id': 'req-e04a9880-309a-497a-b221-5bd3ce2369d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.398199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45de9e974a63488ab0add3c4c86bc919', 'request_id': 'req-e04a9880-309a-497a-b221-5bd3ce2369d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.418182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45de9e974a63488ab0add3c4c86bc919', 'request_id': 'req-e04a9880-309a-497a-b221-5bd3ce2369d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.439776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac420f2-7d99-4328-b402-0d87a03b1f0f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.443222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] The token cache is being invalidate because project 45de9e974a63488ab0add3c4c86bc919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:09.443812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.445317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.448931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 952/1934] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:09 2022] DELETE /identity/v3/projects/45de9e974a63488ab0add3c4c86bc919 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:09.449760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.451963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:09.513315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:09.541228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:09.555011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a9880-309a-497a-b221-5bd3ce2369d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.569316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.570012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.570457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.571235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] PATH_INFO: `/v3/projects/7360dcfaa22649c8a4e5d3bfd72eb8cf/users/2ed9eb60240e48a88e7430049db6d96c/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.576747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.579885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.580195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.580470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.581144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.595564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.644927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 953/1935] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:09 2022] POST /identity/v3/users => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:09.644927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.648213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:09.648879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7360dcfaa22649c8a4e5d3bfd72eb8cf, user_id=2ed9eb60240e48a88e7430049db6d96c, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.651571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.675065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3f57-9a81-4d49-b02f-a84fad687905 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.680705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:09.681646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.681925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.682198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] PATH_INFO: `/v3/users/80f383a49b0441cda4d8dae97f46584c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.683454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f383a49b0441cda4d8dae97f46584c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:09.685528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:09.709082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 983/1936] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:09 2022] PUT /identity/v3/projects/7360dcfaa22649c8a4e5d3bfd72eb8cf/users/2ed9eb60240e48a88e7430049db6d96c/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:09.710008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:09.767025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f383a49b0441cda4d8dae97f46584c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.767698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f383a49b0441cda4d8dae97f46584c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.777060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f383a49b0441cda4d8dae97f46584c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.791466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f383a49b0441cda4d8dae97f46584c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.792539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-902b8154-986d-4226-b446-060336c6db59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f383a49b0441cda4d8dae97f46584c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:09.816478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 954/1937] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:09 2022] DELETE /identity/v3/users/80f383a49b0441cda4d8dae97f46584c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:09.818066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:09.818520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:09.818803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:09.872215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43700-acd8-4d80-8d23-f30bd8800ef0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:09.904963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] MFA Rules not processed for user `2ed9eb60240e48a88e7430049db6d96c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:09.906297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.069984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 955/1938] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:10.075700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.077409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.095074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.095734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.096190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.096508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.097251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.099692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.111591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 984/1939] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:10.116344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.120479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.139914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:10.153415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddee86-2a82-4943-9761-9bdc54757d1e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.172109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632990-8225-443f-b296-781315749d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.173068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632990-8225-443f-b296-781315749d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.174539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632990-8225-443f-b296-781315749d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.175267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632990-8225-443f-b296-781315749d11 admin admin] PATH_INFO: `/v3/projects/4d7f5028030e41bbb47c66df03957157/users/31c10ba968534a47921f5c30a6f30026/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.197167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf5de3-37d0-44e6-a6fa-77a34b51eb0c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.205270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.206245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.206663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.207802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.208709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.208836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0632990-8225-443f-b296-781315749d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f5028030e41bbb47c66df03957157, user_id=31c10ba968534a47921f5c30a6f30026, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.211292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.211550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0632990-8225-443f-b296-781315749d11 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.235867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 985/1940] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:10 2022] PUT /identity/v3/projects/4d7f5028030e41bbb47c66df03957157/users/31c10ba968534a47921f5c30a6f30026/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:10.242045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632990-8225-443f-b296-781315749d11 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.263837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 956/1941] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:10 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:10.268023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.372850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:10.386290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632990-8225-443f-b296-781315749d11 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.418452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baddc49-8452-4d1d-a9d9-95e5fcd233bc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.430025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.430778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.431065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.431339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.432150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.434083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.476755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 957/1942] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:10 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:10.480920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.495838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1acf11-8514-4f02-b564-d3e8ee81e67b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.505016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.506422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.506947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.509091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.510947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.513450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd272-47bb-4639-9ac9-a6bd382e19ed admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.535056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 958/1943] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:10.537958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.539141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.539761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.542723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.543684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.544049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.544411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.545134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.546861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.562809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 986/1944] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:10.562809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.574863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665d15c-4ef1-4bd2-88e1-37fa7804225d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.588125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.589183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.590396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.591527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] PATH_INFO: `/v3/projects/a0524a06ac934abf8e73bf023a68b9e5/users/a5f3cc2a708d4026bf79022e56dea3af/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.623045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0524a06ac934abf8e73bf023a68b9e5, user_id=a5f3cc2a708d4026bf79022e56dea3af, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.637213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.641763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] MFA Rules not processed for user `b1c18aadf10d46018ee874f47fb9b9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:10.643984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.670985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 987/1945] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:10 2022] PUT /identity/v3/projects/a0524a06ac934abf8e73bf023a68b9e5/users/a5f3cc2a708d4026bf79022e56dea3af/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:10.672235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.678022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.790574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 959/1946] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:10.792086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.799889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071a65-9080-42c7-9669-846ca44560ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.807931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.808597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.809397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370b598-965c-4c45-932a-e324b6bd7afb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.809513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.809713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] PATH_INFO: `/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.810873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8ece1d5feb49b4b9fc11b5acf91c32)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.812830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.815787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:10.817692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:10.817976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:10.818255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] PATH_INFO: `/v3/projects/4d7f5028030e41bbb47c66df03957157/users/31c10ba968534a47921f5c30a6f30026/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:10.820267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f5028030e41bbb47c66df03957157, user_id=31c10ba968534a47921f5c30a6f30026, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:10.822988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:10.837494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 960/1947] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:10 2022] PUT /identity/v3/projects/4d7f5028030e41bbb47c66df03957157/users/31c10ba968534a47921f5c30a6f30026/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:10.838529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.865809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:10.882923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'request_id': 'req-e5bca64d-d147-4b24-938b-6acd334921f4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:10.892527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'request_id': 'req-e5bca64d-d147-4b24-938b-6acd334921f4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:10.901284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8ece1d5feb49b4b9fc11b5acf91c32', 'request_id': 'req-e5bca64d-d147-4b24-938b-6acd334921f4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:10.916747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] The token cache is being invalidate because project 4c8ece1d5feb49b4b9fc11b5acf91c32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:10.917073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:10.917755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:10.919264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 988/1948] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:10 2022] DELETE /identity/v3/projects/4c8ece1d5feb49b4b9fc11b5acf91c32 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:10.923253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:10.924647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:10.926395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a62d34-95a9-48a7-bb67-5eb984273741 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:10.931071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:11.002571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.003218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.003897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.004237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.004917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.005946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca64d-d147-4b24-938b-6acd334921f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.006704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de139e7-787a-4071-83ca-97951023bb33 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.012340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.013086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.014243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.014243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.014243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.016201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22e4e1-3c3e-473b-b1da-f5a3f77759ed admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.021668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 961/1949] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:11.022923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.023331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.023632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.055598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 989/1950] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:10 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:11.057522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.057522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.057796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.060011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] MFA Rules not processed for user `31c10ba968534a47921f5c30a6f30026`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:11.061503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.089925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:11.091180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.147756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 962/1951] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:11.149560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.150886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.189914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 990/1952] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:11.189914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.200356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9f404-d4ac-48df-9c17-0962760a1dd5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.205381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.205996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.206275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.206563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.207798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.209156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:11.209740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.240145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 991/1953] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:11.240335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa1163-b196-4b1f-a578-8a6ddce5e988 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.240445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.240522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.250498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.251603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.252044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.252886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] PATH_INFO: `/v3/projects/a0524a06ac934abf8e73bf023a68b9e5/users/a5f3cc2a708d4026bf79022e56dea3af/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.255957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0524a06ac934abf8e73bf023a68b9e5, user_id=a5f3cc2a708d4026bf79022e56dea3af, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.258671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f60797-c0a4-40cb-a56d-6dadaf489ecd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.269796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 963/1954] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:11 2022] PUT /identity/v3/projects/a0524a06ac934abf8e73bf023a68b9e5/users/a5f3cc2a708d4026bf79022e56dea3af/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:11.271020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.271309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.271610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.298189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:11.299396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.312352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bef51-3174-4f0a-9a51-c6bc9bc49cc7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.338719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:11.360825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 964/1955] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:11.361761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.363746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.364373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.364708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.364987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.372043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a40b35-75cd-41c7-9b9f-aef72cba550c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.376733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.377319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.377597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.377871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.378508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.380805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69261465-a468-419b-95b8-d4ccadea2013 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.391086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 965/1956] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:11.392332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.392628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.392891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.421716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] MFA Rules not processed for user `a5f3cc2a708d4026bf79022e56dea3af`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:11.422993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.432011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.433454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.444365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 992/1957] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:11.446834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.466165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8714-90ec-4431-ad9f-e2619e134064 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.477572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.479379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.479379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.479379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.479685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:11.480304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.482623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.507400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 993/1958] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:11.507400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.516118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64f05c-bbed-4300-b59a-16b40fb22846 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.518713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 966/1959] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:11.524061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.524668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.525479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.525829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.526186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] PATH_INFO: `/v3/projects/68f36d31a8bc4b59816263390a5bb46e/users/3c84c02d78624ad990a3d424273f6e11/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.539138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013f49-b100-4b94-98f3-39d51bf147b8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.544403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.545217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.545466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.546045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.546921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.548395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f36d31a8bc4b59816263390a5bb46e, user_id=3c84c02d78624ad990a3d424273f6e11, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.548949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.552430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.586702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 994/1960] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:11 2022] PUT /identity/v3/projects/68f36d31a8bc4b59816263390a5bb46e/users/3c84c02d78624ad990a3d424273f6e11/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:11.587274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.595841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 967/1961] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:11.599778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.657705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:11.676112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24442fd-52f9-4f29-8a41-fd8cec24c744 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.702465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4abf8-5705-4142-beb3-be848a3fc618 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.707828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.708453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.708743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.709009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.709994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.711734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.727804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 968/1962] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:11.727804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.739667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871af66-48c5-4a8c-af3c-d540e2994766 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.741814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.742472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.742763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.743038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.744830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.745393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.745669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.745946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.746572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.748929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.754951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:11.781851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 969/1963] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:11 2022] POST /identity/v3/users => generated 294 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:11.783184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.785360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.798164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045c945-b558-433e-b4ab-217416662f07 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.803777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.804571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.804818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.805203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.807172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:11.826188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.827863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.854503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 995/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:11.855324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.866581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a261-afe7-424c-8c72-5ff6720ba3ef service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:11.872151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:11.872818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:11.873101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:11.873379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] PATH_INFO: `/v3/projects/68f36d31a8bc4b59816263390a5bb46e/users/3c84c02d78624ad990a3d424273f6e11/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:11.876862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f36d31a8bc4b59816263390a5bb46e, user_id=3c84c02d78624ad990a3d424273f6e11, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.881873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.895139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 996/1965] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:11 2022] PUT /identity/v3/projects/68f36d31a8bc4b59816263390a5bb46e/users/3c84c02d78624ad990a3d424273f6e11/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:11.896281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:11.916638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:11.920012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:11.947842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 970/1966] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:12.000375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60430-3e7c-4f5a-b718-9c87815d2127 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.011279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.012208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.012485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.012792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.013404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.018642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.044765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 997/1967] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:12.045828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.065276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45cbc0-3492-4b9d-ae4f-97c1a27c9de9 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.084492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.087041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.087700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.087992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.088832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.092437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.107847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.125347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 971/1968] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:12.125636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2814-15c3-43c1-a28e-b992af7a0990 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.130516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.139897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.140732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.142634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b08c-b9dc-4022-a9a0-fb2693fc6d1c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.143062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.143497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] PATH_INFO: `/v3/users/2ed9eb60240e48a88e7430049db6d96c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.148471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed9eb60240e48a88e7430049db6d96c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.148609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.149286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.149576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.149856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] PATH_INFO: `/v3/projects/a06717b71dd14ab98351879165c42f95/users/11c2deb2dea04016b23546afda52157b/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.153300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.171466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06717b71dd14ab98351879165c42f95, user_id=11c2deb2dea04016b23546afda52157b, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.175020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553371b4-8c84-4386-abe8-355eba31d731 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.223914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 972/1969] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:12 2022] PUT /identity/v3/projects/a06717b71dd14ab98351879165c42f95/users/11c2deb2dea04016b23546afda52157b/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:12.237007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.238825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.240157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.267358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed9eb60240e48a88e7430049db6d96c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:12.268997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed9eb60240e48a88e7430049db6d96c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:12.299979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed9eb60240e48a88e7430049db6d96c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:12.301707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] MFA Rules not processed for user `11c2deb2dea04016b23546afda52157b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:12.305337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:12.344253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed9eb60240e48a88e7430049db6d96c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:12.345148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ef9b914d-b78f-48ee-a3d3-349d4deb7550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed9eb60240e48a88e7430049db6d96c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:12.398537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 998/1970] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:12 2022] DELETE /identity/v3/users/2ed9eb60240e48a88e7430049db6d96c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:12.404813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.406179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.406471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.423614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:12.430110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 973/1971] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:12.435444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.436329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:12.440539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:12.500236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:12.502695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:12.636514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43455fc-97b9-42ee-8a0f-882bb312f70f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.641127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 999/1972] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:12.657103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.658437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.660962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.661263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.663692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.666632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.668179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:12.670155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.753303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 974/1973] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:12 2022] POST /identity/v3/projects => generated 387 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:12.760005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.788737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3411b8-0b21-44b1-b053-9ab2708c9e1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.875447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d48f1-2e8b-44ec-a865-80fae4653480 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.878832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.880407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.880797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.881290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.883209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.884097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.884589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.885125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.886047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.888397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.896346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.899003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.922255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 975/1974] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:12 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:12.922427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1000/1975] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:12.932836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.945991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4a99-3098-4fa7-9289-ca102e2fe731 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.951206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.952509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.952985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.953793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:12.954078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:12.957605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:12.973542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 976/1976] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:12.977230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:12.987456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db888d7-5b5a-4668-8a2e-ef6b7a3eeb44 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:12.993139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:12.993842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:12.994126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:12.994405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] PATH_INFO: `/v3/projects/3df78e091d04493fbfc2f718687189d2/users/94ca81f4e5a747a6a24a9419359ab1a3/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.013557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df78e091d04493fbfc2f718687189d2, user_id=94ca81f4e5a747a6a24a9419359ab1a3, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.016348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.032470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1001/1977] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:12 2022] PUT /identity/v3/projects/3df78e091d04493fbfc2f718687189d2/users/94ca81f4e5a747a6a24a9419359ab1a3/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:13.037320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.049930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6589cb-ecb9-431c-b6f8-d37c35a40abd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.130899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.131845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.132281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.132976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.134482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.139653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64fd7-bdc3-4d26-9b93-5ef0ef86cbea admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.144008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:13.157108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 977/1978] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:13.161999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.177632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbad6-18c1-4a6e-9e19-d1e49db252bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.192957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.193680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.193938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.194844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] PATH_INFO: `/v3/projects/3df78e091d04493fbfc2f718687189d2/users/94ca81f4e5a747a6a24a9419359ab1a3/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.200118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df78e091d04493fbfc2f718687189d2, user_id=94ca81f4e5a747a6a24a9419359ab1a3, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.203569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73241-5db8-4245-9988-7c528c5e861c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.231598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1002/1979] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:13 2022] PUT /identity/v3/projects/3df78e091d04493fbfc2f718687189d2/users/94ca81f4e5a747a6a24a9419359ab1a3/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:13.239181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.239469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.239767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.276207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] MFA Rules not processed for user `94ca81f4e5a747a6a24a9419359ab1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:13.277665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.364195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.364529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.364889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.422697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 978/1980] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:13.429634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.433638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.467085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] MFA Rules not processed for user `3c84c02d78624ad990a3d424273f6e11`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:13.472426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.509874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:13.593960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75987b0-156e-4765-b08d-4105c7a631d0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.603704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.604764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.605152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.605882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.620452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.629662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.642383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1003/1981] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:26:13.643667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.646052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.654754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 979/1982] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:13.738461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.743902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e8ff7-ea7e-4ca8-b665-6a8249d9976b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.747191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee634-6d7d-498d-8fa9-62aec5949944 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.748980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.749821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.750121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.750395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] PATH_INFO: `/v3/projects/7360dcfaa22649c8a4e5d3bfd72eb8cf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.751888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7360dcfaa22649c8a4e5d3bfd72eb8cf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.752588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.753390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.754801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.754801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.755159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.756578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:13.810902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7360dcfaa22649c8a4e5d3bfd72eb8cf', 'request_id': 'req-3be2e0bb-0e69-4cfc-aed2-7286ab992859'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:13.823207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7360dcfaa22649c8a4e5d3bfd72eb8cf', 'request_id': 'req-3be2e0bb-0e69-4cfc-aed2-7286ab992859'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:13.824021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.826086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:13.829915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7360dcfaa22649c8a4e5d3bfd72eb8cf', 'request_id': 'req-3be2e0bb-0e69-4cfc-aed2-7286ab992859'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:13.838508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 980/1983] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:13.859326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] The token cache is being invalidate because project 7360dcfaa22649c8a4e5d3bfd72eb8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:13.863356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:13.864318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:13.873219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1004/1984] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:13 2022] DELETE /identity/v3/projects/7360dcfaa22649c8a4e5d3bfd72eb8cf => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:13.874351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:13.889501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68712d2-3fe5-4547-af39-94575f5ad2ec service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:13.916181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:13.978299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:13.978955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:13.979724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:13.979724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] PATH_INFO: `/v3/users/a5f3cc2a708d4026bf79022e56dea3af` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:13.982866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f3cc2a708d4026bf79022e56dea3af)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:13.987428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:14.070733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f3cc2a708d4026bf79022e56dea3af'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.070733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f3cc2a708d4026bf79022e56dea3af'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.084891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f3cc2a708d4026bf79022e56dea3af'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.109324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f3cc2a708d4026bf79022e56dea3af'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.110564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5adb0da1-a610-4925-8a04-c663829d3c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f3cc2a708d4026bf79022e56dea3af'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.140988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 981/1985] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:13 2022] DELETE /identity/v3/users/a5f3cc2a708d4026bf79022e56dea3af => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:14.697361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:14.699582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:14.724142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2e0bb-0e69-4cfc-aed2-7286ab992859 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:14.822194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:14.860655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:14.861672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:14.862096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:14.863734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] PATH_INFO: `/v3/projects/a0524a06ac934abf8e73bf023a68b9e5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:14.866049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0524a06ac934abf8e73bf023a68b9e5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:14.868658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:14.943758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0524a06ac934abf8e73bf023a68b9e5', 'request_id': 'req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.954442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0524a06ac934abf8e73bf023a68b9e5', 'request_id': 'req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.964684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0524a06ac934abf8e73bf023a68b9e5', 'request_id': 'req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.979711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] The token cache is being invalidate because project a0524a06ac934abf8e73bf023a68b9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:14.979962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.980659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31af183e-9a0b-4d7b-b25b-d3d5b930cd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:14.982493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1005/1986] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:14 2022] DELETE /identity/v3/projects/a0524a06ac934abf8e73bf023a68b9e5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:14.987198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:14.987753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:14.987840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.017162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.019055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.098423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 982/1987] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:15.108627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.108627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.109188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.133866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.135415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.153917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1006/1988] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:15.155472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.167192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38649a3c-3e0f-4c94-ac0c-c57189d24dc7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.172243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.173168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.173457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.173733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.174406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:15.174755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.177071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.193616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 983/1989] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:15.195007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.207311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0283-3dc8-410b-acfb-68e7f0b363fe None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.224706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.225341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.225626 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.225905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.226560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.228567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.267475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1007/1990] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:15.270515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.283015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b7722-96de-49b7-a40d-12c8060f36d4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.321682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.323483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.329540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:15.393478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.394958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.395356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.397330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.400542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.402467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.462965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4fa3-514d-4500-b125-213739481aad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.464898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 984/1991] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:15.469015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.471350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.474173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.474593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.475105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] PATH_INFO: `/v3/users/11c2deb2dea04016b23546afda52157b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.476621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c2deb2dea04016b23546afda52157b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.480539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdd5be-89d7-4077-8574-109dfe37a1df admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.493143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.493981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.494429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.494855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.495695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.495924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.497804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.510955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 985/1992] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:15.514869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.527083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5ff72-faab-44e0-9488-cbba55b0ef73 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.532347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.533293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.533652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.534098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.535300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.537983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.573252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 986/1993] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:15.577790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.590785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:15.592776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9b75c-0809-4194-bbe1-5851cab914ab admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.601634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c2deb2dea04016b23546afda52157b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:15.602447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c2deb2dea04016b23546afda52157b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:15.605009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.606073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.618271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c2deb2dea04016b23546afda52157b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:15.620738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.621059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] PATH_INFO: `/v3/projects/839400f909dc4caebb4ff9cfabed0d7c/users/d5476aacdb0b4015827a67be07e744ce/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.638322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c2deb2dea04016b23546afda52157b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:15.639355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c2deb2dea04016b23546afda52157b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:15.664067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=839400f909dc4caebb4ff9cfabed0d7c, user_id=d5476aacdb0b4015827a67be07e744ce, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.669414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5257-a550-4e75-8427-373576d7c20c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.677136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1008/1994] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:15 2022] DELETE /identity/v3/users/11c2deb2dea04016b23546afda52157b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:15.678868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.680940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.697532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 987/1995] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:15 2022] PUT /identity/v3/projects/839400f909dc4caebb4ff9cfabed0d7c/users/d5476aacdb0b4015827a67be07e744ce/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:15.698855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d3fc-3a5a-4ad7-8ec2-ebc4b7344b3a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.699249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d3fc-3a5a-4ad7-8ec2-ebc4b7344b3a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.699633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d3fc-3a5a-4ad7-8ec2-ebc4b7344b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.724763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-caf7d3fc-3a5a-4ad7-8ec2-ebc4b7344b3a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.726209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-caf7d3fc-3a5a-4ad7-8ec2-ebc4b7344b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.766659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a68e1-f3e3-4d04-a762-e2095f69d22c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.812683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 988/1996] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:15.814227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.814777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.815055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.830857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:15.832586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:15.833277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:15.833848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:15.837819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.872836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] MFA Rules not processed for user `d5476aacdb0b4015827a67be07e744ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:15.874337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.924889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:15.928427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:15.940551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1009/1997] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:15.941774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.947865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.963244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 989/1998] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:15.968637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:15.972108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:15.986550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:15.992818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde3f48-d016-4ea4-9039-a559d5edc129 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:15.999103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.000485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.000770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.001272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.002254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.005661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.010603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:16.054494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 990/1999] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:15 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:16.058426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.067133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc29b7-9aac-4707-9e44-445f0638c802 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.166365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.167240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.167752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.168116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.241604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46778a61-0225-45ae-8167-f9a22edfcb91 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.257357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.257915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.258060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.258255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.258530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.259173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.260630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.261124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.286316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1010/2000] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:16.304833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 991/2001] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:16 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:16.309209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.327445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854726-5544-4e09-9147-41bb7bca5283 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.340166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.340927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.341304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.341680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.342877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.351027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.379922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.385013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1011/2002] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:16.391065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee66846-41a8-458e-b42d-6cc485e7c09d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.397354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.398309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.398570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.398845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] PATH_INFO: `/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b/users/232b41cbb1d44607b06f865239b392cd/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.422431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69600bf9ab0d46dd9dce1d6a6974480b, user_id=232b41cbb1d44607b06f865239b392cd, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.426367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.446746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 992/2003] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:16 2022] PUT /identity/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b/users/232b41cbb1d44607b06f865239b392cd/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:16.448644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.466934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99446e43-bb0d-4594-9a66-9839358b53fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.579437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.580422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.580700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.581447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.582199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.584030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70bc5f-9bd5-41e9-87c8-da6b006be1ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.596026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1012/2004] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:16.600369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.613676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a0c24-158d-4bca-85ba-4ac967a7d90c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.620062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.620866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.621245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.621610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] PATH_INFO: `/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b/users/232b41cbb1d44607b06f865239b392cd/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.635058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=69600bf9ab0d46dd9dce1d6a6974480b, user_id=232b41cbb1d44607b06f865239b392cd, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.638463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.658525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 993/2005] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:16 2022] PUT /identity/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b/users/232b41cbb1d44607b06f865239b392cd/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:16.669518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.669748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.669960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.701463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] MFA Rules not processed for user `232b41cbb1d44607b06f865239b392cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:16.703173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:16.710000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:16.757835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.759772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:16.800581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1013/2006] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:16.814725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.817491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:16.842795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601011f0-4662-4848-9de0-6dd8e74c35ed admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.848609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.849477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.849918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.850341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] PATH_INFO: `/v3/projects/a06717b71dd14ab98351879165c42f95` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.851683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06717b71dd14ab98351879165c42f95)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:16.854231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:16.896009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9454de1-4ca5-4bda-8fa7-75999a785326 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:16.899583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:16.917296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06717b71dd14ab98351879165c42f95', 'request_id': 'req-052027b6-9cad-4703-8c7a-b9cee4b8f8df'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:16.927007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06717b71dd14ab98351879165c42f95', 'request_id': 'req-052027b6-9cad-4703-8c7a-b9cee4b8f8df'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:16.934322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06717b71dd14ab98351879165c42f95', 'request_id': 'req-052027b6-9cad-4703-8c7a-b9cee4b8f8df'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:16.946546 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] The token cache is being invalidate because project a06717b71dd14ab98351879165c42f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:16.946836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:16.947503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:16.949285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 994/2007] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:16 2022] DELETE /identity/v3/projects/a06717b71dd14ab98351879165c42f95 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:16.952932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:16.954294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:16.958027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:16.958857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:16.959235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:16.959710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:16.961525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:17.027674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052027b6-9cad-4703-8c7a-b9cee4b8f8df admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:17.029439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.030952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:17.032694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:17.033417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.033757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.034120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] PATH_INFO: `/v3/users/94ca81f4e5a747a6a24a9419359ab1a3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.037193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ca81f4e5a747a6a24a9419359ab1a3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.039042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:17.042600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1014/2008] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:17.104317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ca81f4e5a747a6a24a9419359ab1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.105677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ca81f4e5a747a6a24a9419359ab1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.121061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ca81f4e5a747a6a24a9419359ab1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.135840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ca81f4e5a747a6a24a9419359ab1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.136791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7e707fad-6aaf-49c2-90fa-6ece2990a964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ca81f4e5a747a6a24a9419359ab1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.151973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 995/2009] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:16 2022] DELETE /identity/v3/users/94ca81f4e5a747a6a24a9419359ab1a3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:17.489181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:17.511101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e33022-1eca-44bb-9e5f-c1d9c5b53363 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:17.523184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:17.592376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:17.593006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.593283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.593554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] PATH_INFO: `/v3/projects/3df78e091d04493fbfc2f718687189d2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.594668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df78e091d04493fbfc2f718687189d2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.596908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:17.663163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df78e091d04493fbfc2f718687189d2', 'request_id': 'req-e75d0f42-867b-4c05-ab69-675e273b8fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.672751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df78e091d04493fbfc2f718687189d2', 'request_id': 'req-e75d0f42-867b-4c05-ab69-675e273b8fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.679467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df78e091d04493fbfc2f718687189d2', 'request_id': 'req-e75d0f42-867b-4c05-ab69-675e273b8fe3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.691423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] The token cache is being invalidate because project 3df78e091d04493fbfc2f718687189d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:17.691891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.692568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e75d0f42-867b-4c05-ab69-675e273b8fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:17.694262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1015/2010] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:17 2022] DELETE /identity/v3/projects/3df78e091d04493fbfc2f718687189d2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:17.699172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.699461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.699768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.723443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:17.724801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:17.768650 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:17.788728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 996/2011] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:17.796830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.797003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.797147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.817572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:17.818717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:17.826978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1016/2012] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:17.831458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:17.841497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecbe10-900a-4730-8c6b-a668ddf62d39 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:17.847771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:17.848755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.849048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.850135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.851572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:17.852006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.853487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:17.870300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 997/2013] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:17.871111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:17.884531 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0a9cd-1e36-48b1-a8dd-bc9881726812 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:17.889494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:17.890102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.890379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.890656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.891435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.893396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:17.922240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1017/2014] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:17 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:17.927046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:17.937136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf120-553d-4ba3-9d9a-bd36a5904ac0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:17.992795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:17.993402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:17.993679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:17.993951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:17.994621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:17.996566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.020212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 998/2015] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:17 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:18.024395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:18.035151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26685f5-ce99-4c23-87ae-7827c45706d8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:18.039907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:18.040507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.040787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.041061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.041704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:18.043287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.053932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1018/2016] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:18.058099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:18.069114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400c9f-4c52-46fb-9578-3e69f6660dda admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:18.073714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:18.074311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.074592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.074884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.075562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:18.077156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a321e0-2b54-4d9b-8e87-032b3c695e6e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.088523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 999/2017] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:18.092687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:18.103451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e3bb2-b875-4c61-a0b9-20b9c73fc3b6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:18.108236 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:18.108878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.109156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.109486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] PATH_INFO: `/v3/projects/9231500f143c462aaeed8c48a06d1760/users/c822832b45ce4825bc547e5a4c32bdc3/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.123774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9231500f143c462aaeed8c48a06d1760, user_id=c822832b45ce4825bc547e5a4c32bdc3, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:18.126459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.138627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1019/2018] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:18 2022] PUT /identity/v3/projects/9231500f143c462aaeed8c48a06d1760/users/c822832b45ce4825bc547e5a4c32bdc3/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:18.150531 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.150812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.151081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.179918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] MFA Rules not processed for user `c822832b45ce4825bc547e5a4c32bdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:18.181136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:18.244646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1000/2019] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:18.256321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:18.258059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:18.301924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:18.316944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a2f14-4e3e-49ec-bb1f-2eac9461fd8e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:18.320685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:18.321227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:18.321505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:18.321776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:18.329935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:18.331479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5d7fc-bcdb-4905-ba32-261b044368ec service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:18.340402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1020/2020] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:28.241709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:28.336435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:43.695940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:43.696432 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:43.782994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc828a07-c6d1-4343-87fb-67c4c459a37c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:43.788583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:43.789222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:43.789502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:43.789776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] PATH_INFO: `/v3/users/b7a8294318064eb6929055e52b71309d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:43.790977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a8294318064eb6929055e52b71309d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:43.792912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:43.850553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a8294318064eb6929055e52b71309d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:43.852082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a8294318064eb6929055e52b71309d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:43.860775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a8294318064eb6929055e52b71309d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:43.874819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a8294318064eb6929055e52b71309d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:43.875602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a8294318064eb6929055e52b71309d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:43.891752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1001/2021] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:43 2022] DELETE /identity/v3/users/b7a8294318064eb6929055e52b71309d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:43.894020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:43.894568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:43.894661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:43.915779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:43.916956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:43.988268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1021/2022] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:43.998530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:44.000150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:44.077825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a766f-a360-4f8f-b685-8439837a1b7a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:44.082460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:44.083246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:44.083581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:44.083873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:44.093557 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:44.094979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:44.108389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1002/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:44.419839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:44.421309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:44.437940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e2c44-727e-4f41-8886-fa509bfb3b50 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:44.442892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:44.443550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:44.443886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:44.444165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] PATH_INFO: `/v3/projects/bdfe4124a70c4da78a6ee17b150d5523` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:44.445299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfe4124a70c4da78a6ee17b150d5523)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:44.447322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:44.500086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfe4124a70c4da78a6ee17b150d5523', 'request_id': 'req-0eca9e87-90ec-4718-9f46-a2fe85815dad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.508057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfe4124a70c4da78a6ee17b150d5523', 'request_id': 'req-0eca9e87-90ec-4718-9f46-a2fe85815dad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.513721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfe4124a70c4da78a6ee17b150d5523', 'request_id': 'req-0eca9e87-90ec-4718-9f46-a2fe85815dad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.525677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] The token cache is being invalidate because project bdfe4124a70c4da78a6ee17b150d5523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:44.525966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.526605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.528112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1022/2024] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:44 2022] DELETE /identity/v3/projects/bdfe4124a70c4da78a6ee17b150d5523 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:44.532328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:44.544860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa2eaf-b98f-48d1-96cd-a679deeebf6c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:44.563373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:44.607218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:44.607887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:44.608240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:44.608596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] PATH_INFO: `/v3/users/31c10ba968534a47921f5c30a6f30026` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:44.609904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c10ba968534a47921f5c30a6f30026)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:44.611832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:44.669002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c10ba968534a47921f5c30a6f30026'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.669666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c10ba968534a47921f5c30a6f30026'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.672016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:44.673486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:44.677047 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c10ba968534a47921f5c30a6f30026'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.689332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c10ba968534a47921f5c30a6f30026'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.689950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c10ba968534a47921f5c30a6f30026'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:44.704005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1003/2025] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:44 2022] DELETE /identity/v3/users/31c10ba968534a47921f5c30a6f30026 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:44.744135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca9e87-90ec-4718-9f46-a2fe85815dad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:44.788141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:44.801319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:44.801924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:44.802200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:44.802469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:44.804225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:44.880064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:44.881507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53650d8-2b93-42b1-91c7-af6ff23ac2a8 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:44.894527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1023/2026] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:26:45.021906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.023778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:45.130734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731beb1f-b5ac-4663-81fb-e29385969c51 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.138160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.139046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.139398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.140514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] PATH_INFO: `/v3/projects/4d7f5028030e41bbb47c66df03957157` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.141746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7f5028030e41bbb47c66df03957157)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.144550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.243888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:45.243888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7f5028030e41bbb47c66df03957157', 'request_id': 'req-edc08def-da7a-49ad-80d3-d101903f9770'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:45.257639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7f5028030e41bbb47c66df03957157', 'request_id': 'req-edc08def-da7a-49ad-80d3-d101903f9770'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:45.268845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7f5028030e41bbb47c66df03957157', 'request_id': 'req-edc08def-da7a-49ad-80d3-d101903f9770'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:45.284845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] The token cache is being invalidate because project 4d7f5028030e41bbb47c66df03957157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:45.285413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:45.286332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-edc08def-da7a-49ad-80d3-d101903f9770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:45.292583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1004/2027] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:45 2022] DELETE /identity/v3/projects/4d7f5028030e41bbb47c66df03957157 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:45.294389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.294701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.295355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.337388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:45.338896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:45.426737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1024/2028] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:45.434657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714c43-0f83-4380-a084-451befd1889b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.435017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714c43-0f83-4380-a084-451befd1889b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.435555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714c43-0f83-4380-a084-451befd1889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.473089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-52714c43-0f83-4380-a084-451befd1889b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:45.477097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-52714c43-0f83-4380-a084-451befd1889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:45.499345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1005/2029] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:45.506121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.527089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2d0f0-a3c4-4343-8520-e2e2aac022a3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.537231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.537868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.538140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.538418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.539292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:45.539397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.541302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.559676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1025/2030] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:45.560672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52714c43-0f83-4380-a084-451befd1889b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.572070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52714c43-0f83-4380-a084-451befd1889b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.592444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.593075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.593361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.593639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.594322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.598889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.666016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1006/2031] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:45 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:45.667128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.680704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b25f-e390-454d-866c-fe83907f7196 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.753987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.754843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.755229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.755708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.756544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.758628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.785431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1026/2032] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:45 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:45.789342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.801957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8a5c-fe3f-444b-a134-0d7faabd0def admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.807174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.807911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.808194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.808463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.809125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.810714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.822172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1007/2033] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:45.826023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.837252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989eda-93ce-498c-b6b4-8c7c73efd4e8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.843117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.843763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.844049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.844324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.844987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.846651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e5d73-7fc3-4c76-a09d-786a3621eff2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.848527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:45.860916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1027/2034] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:45.865871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:45.876327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6ef85-e113-43fd-924a-15740ce15e2f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:45.881415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:45.882116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.882389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.882657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] PATH_INFO: `/v3/projects/9ffb0b1ab3c04813afb8ab54c0e12292/users/cae1308c80194fa4b66a232cf17ee6a0/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.899256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffb0b1ab3c04813afb8ab54c0e12292, user_id=cae1308c80194fa4b66a232cf17ee6a0, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:45.901979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:45.915716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1008/2035] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:45 2022] PUT /identity/v3/projects/9ffb0b1ab3c04813afb8ab54c0e12292/users/cae1308c80194fa4b66a232cf17ee6a0/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:26:45.927009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:45.927479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:45.927635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:45.955052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] MFA Rules not processed for user `cae1308c80194fa4b66a232cf17ee6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:45.956933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:46.021278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1028/2036] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:46.025676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.087797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1e80-bd67-4370-b986-57ab34d36ab9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.092628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.093290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.093572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.093847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.094516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.097027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.123237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1009/2037] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:46 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:46.127216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.138220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8567adc-ca6a-493b-ba1a-424e03b879ec None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.194197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.194801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.195086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.195368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.196058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.198054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.219940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1029/2038] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:46 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:46.224477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.235856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120545a4-4a6d-4f48-8be4-efdabab2432b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.241578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.242192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.242635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.242912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.243583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.245195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.259698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1010/2039] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:46.261375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.272425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadbd3d-f997-4b97-92b3-36670fe13fa7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.277066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.277724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.278004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.278898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] PATH_INFO: `/v3/projects/c79b749bc5c64fe687bc8275e7821627/users/16469f3a17b14fe6b4f5736f155c6ab5/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.294905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79b749bc5c64fe687bc8275e7821627, user_id=16469f3a17b14fe6b4f5736f155c6ab5, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.297931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.311392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1030/2040] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:46 2022] PUT /identity/v3/projects/c79b749bc5c64fe687bc8275e7821627/users/16469f3a17b14fe6b4f5736f155c6ab5/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:46.315885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.326020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdf6da-c26c-47c6-b976-79dab39b8a7a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.374155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:46.398467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.399089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.399373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.399689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.400391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.401971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3bce4-2a70-4ee9-85b1-f49ed06649d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.414166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1011/2041] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:46.418195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.430013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b8c7-575f-4e44-a4bb-36ef04d1b8c1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.435721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.436494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.436780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.437090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] PATH_INFO: `/v3/projects/c79b749bc5c64fe687bc8275e7821627/users/16469f3a17b14fe6b4f5736f155c6ab5/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.439325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79b749bc5c64fe687bc8275e7821627, user_id=16469f3a17b14fe6b4f5736f155c6ab5, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.443075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.454991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1031/2042] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:46 2022] PUT /identity/v3/projects/c79b749bc5c64fe687bc8275e7821627/users/16469f3a17b14fe6b4f5736f155c6ab5/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:46.465588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.465878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.466161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.499145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] MFA Rules not processed for user `16469f3a17b14fe6b4f5736f155c6ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:46.500921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:46.597818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1012/2043] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:46.609265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:46.612429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:46.668413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:46.696765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc77a-a882-4fc0-90d4-df1676eb5ee4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:46.700812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:46.701538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:46.701898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:46.702258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:46.713265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:46.714716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06938b-f047-4fb1-837a-4b737f84a425 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:46.727675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1032/2044] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:47.098834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:47.160780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b88d-e098-4b9f-a9b5-83909e4d64b8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:47.165180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:47.165793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:47.166075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:47.166344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] PATH_INFO: `/v3/users/cae1308c80194fa4b66a232cf17ee6a0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:47.167443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae1308c80194fa4b66a232cf17ee6a0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:47.169361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:47.214854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae1308c80194fa4b66a232cf17ee6a0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.215499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae1308c80194fa4b66a232cf17ee6a0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.222887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae1308c80194fa4b66a232cf17ee6a0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.229833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:47.236176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae1308c80194fa4b66a232cf17ee6a0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.237067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae1308c80194fa4b66a232cf17ee6a0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.249366 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1013/2045] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:47 2022] DELETE /identity/v3/users/cae1308c80194fa4b66a232cf17ee6a0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:47.254050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e7db-6afb-4645-af99-713349d2116d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:47.254512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e7db-6afb-4645-af99-713349d2116d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:47.254618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e7db-6afb-4645-af99-713349d2116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:47.274588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3967e7db-6afb-4645-af99-713349d2116d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:47.275793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3967e7db-6afb-4645-af99-713349d2116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:47.341714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1033/2046] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:47.352077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:47.353467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:47.412504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49eec0-04d4-4b57-a640-23550b29b176 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:47.416113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:47.416715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:47.416995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:47.417266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:47.426348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:47.427822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:47.437403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1014/2047] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:47.720875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e7db-6afb-4645-af99-713349d2116d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:47.725135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3967e7db-6afb-4645-af99-713349d2116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:47.769644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e7db-6afb-4645-af99-713349d2116d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:47.776232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:47.777000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:47.777278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:47.777567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] PATH_INFO: `/v3/projects/9ffb0b1ab3c04813afb8ab54c0e12292` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:47.778829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffb0b1ab3c04813afb8ab54c0e12292)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:47.780879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:47.861543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffb0b1ab3c04813afb8ab54c0e12292', 'request_id': 'req-a2659835-ea74-49c1-bd87-4bfced742b1f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.870027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffb0b1ab3c04813afb8ab54c0e12292', 'request_id': 'req-a2659835-ea74-49c1-bd87-4bfced742b1f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.876922 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffb0b1ab3c04813afb8ab54c0e12292', 'request_id': 'req-a2659835-ea74-49c1-bd87-4bfced742b1f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.894778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] The token cache is being invalidate because project 9ffb0b1ab3c04813afb8ab54c0e12292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:47.895048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.896152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:47.897375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1034/2048] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:47 2022] DELETE /identity/v3/projects/9ffb0b1ab3c04813afb8ab54c0e12292 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:47.901515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:47.915887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cd00e-d7ec-4163-a47f-4ac2b8221652 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:48.009767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:48.010467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.010759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.011037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] PATH_INFO: `/v3/users/16469f3a17b14fe6b4f5736f155c6ab5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.017451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16469f3a17b14fe6b4f5736f155c6ab5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:48.022453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:48.114813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16469f3a17b14fe6b4f5736f155c6ab5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.115802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16469f3a17b14fe6b4f5736f155c6ab5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.139747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16469f3a17b14fe6b4f5736f155c6ab5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.160880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16469f3a17b14fe6b4f5736f155c6ab5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.161834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-64e1f664-ef81-4415-990d-08f3c4218414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16469f3a17b14fe6b4f5736f155c6ab5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.188896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1015/2049] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:47 2022] DELETE /identity/v3/users/16469f3a17b14fe6b4f5736f155c6ab5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:48.494349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:48.495937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:48.509725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659835-ea74-49c1-bd87-4bfced742b1f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:48.575157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:48.575830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.576113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.576382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] PATH_INFO: `/v3/projects/c79b749bc5c64fe687bc8275e7821627` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.577507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79b749bc5c64fe687bc8275e7821627)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:48.579426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:48.587455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:48.635643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79b749bc5c64fe687bc8275e7821627', 'request_id': 'req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.643624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79b749bc5c64fe687bc8275e7821627', 'request_id': 'req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.649556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79b749bc5c64fe687bc8275e7821627', 'request_id': 'req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.661571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] The token cache is being invalidate because project c79b749bc5c64fe687bc8275e7821627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:48.661859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.662512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a1dcc84d-73c7-4584-ac37-6f0fd0859c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:48.664239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1035/2050] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:48 2022] DELETE /identity/v3/projects/c79b749bc5c64fe687bc8275e7821627 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:48.668893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.669255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.669602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.679130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:48.692295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:48.693982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:48.755319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1016/2051] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:48.762874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.763254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.763659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.790510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:48.793399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:48.804028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1036/2052] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:48.808280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:48.818850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01652c7-8a24-434e-ab57-666f5f2d26d1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:48.824850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:48.825499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.825872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.826185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.826821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:48.827116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:48.828865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:48.848901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1017/2053] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:26:48.852621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:48.864240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb56b-0bca-4d19-9c5a-498a9919c2f7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:48.870437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:48.871399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.871864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.872383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.873294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:48.875475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:48.915900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1037/2054] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:48 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:26:48.919982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:48.930700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd223ec-83c9-4148-a5ca-c889197c93e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:48.990658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:48.991269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:48.991573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:48.991855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:48.992490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:48.994451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:49.017453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1018/2055] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:48 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:49.021745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:49.032052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e23c00-d440-4558-bee9-41e54a288ffd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:49.037481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:49.038083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:49.038361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:49.038634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:49.039277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:49.040994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:49.051421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1038/2056] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:49.055680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:49.067306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edc8e-b936-49e0-bd4e-acf899d4189c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:49.071971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:49.072576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:49.072854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:49.073126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:49.073772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:49.075347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e21b92-8e30-460f-9263-f44edec2bf86 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:49.086864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1019/2057] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:49.091023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:49.101620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fc6e9-6c09-4513-85d5-88ef670cd4d0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:49.106793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:49.107635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:49.107786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:49.108041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] PATH_INFO: `/v3/projects/e2a264c27225418d9169bb1f1707cb4a/users/8471e8f0923e4723ba148b941c83a377/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:49.124918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a264c27225418d9169bb1f1707cb4a, user_id=8471e8f0923e4723ba148b941c83a377, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:49.128060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:49.141564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1039/2058] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:49 2022] PUT /identity/v3/projects/e2a264c27225418d9169bb1f1707cb4a/users/8471e8f0923e4723ba148b941c83a377/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:49.152848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:49.153333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:49.153333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:49.185890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] MFA Rules not processed for user `8471e8f0923e4723ba148b941c83a377`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:49.187500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:49.277524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:49.283160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1020/2059] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:49.296987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:49.298808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:49.480955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab89b1-856e-4637-811a-e42623fc538e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:49.487422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:49.489133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:49.489511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:49.489845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:49.499979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:49.501534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:49.516798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1040/2060] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:51.867271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:51.869385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:51.936817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625c349-a18d-4651-ad86-6f370bb80ff9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:51.941372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:51.941997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:51.942278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:51.942553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] PATH_INFO: `/v3/users/afd147dfc8c648149712518771f7940c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:51.943701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd147dfc8c648149712518771f7940c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:51.945663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:51.992276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd147dfc8c648149712518771f7940c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:51.992859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd147dfc8c648149712518771f7940c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:51.999985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd147dfc8c648149712518771f7940c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.012160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd147dfc8c648149712518771f7940c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.012796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8822f6eb-10f8-4e70-9c02-085286504012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd147dfc8c648149712518771f7940c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.026015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1021/2061] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:26:51 2022] DELETE /identity/v3/users/afd147dfc8c648149712518771f7940c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:52.223619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.238054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e17ea-eec3-49ed-a71e-7ed34a1e063a service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.249604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:52.300754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.301373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.301651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.301925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] PATH_INFO: `/v3/projects/777c1bff7c024f91b0eafcad7fce29bc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.303023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=777c1bff7c024f91b0eafcad7fce29bc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.304948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.351272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777c1bff7c024f91b0eafcad7fce29bc', 'request_id': 'req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.358528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777c1bff7c024f91b0eafcad7fce29bc', 'request_id': 'req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.364836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777c1bff7c024f91b0eafcad7fce29bc', 'request_id': 'req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.375681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] The token cache is being invalidate because project 777c1bff7c024f91b0eafcad7fce29bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:26:52.375967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.376609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2253ae8-dfa9-4e16-ae33-d3a93c2de22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:26:52.378395 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1041/2062] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:26:52 2022] DELETE /identity/v3/projects/777c1bff7c024f91b0eafcad7fce29bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:52.383349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.383678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.383954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.403773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:52.404963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:52.465476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1022/2063] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:52.473680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.473964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.474238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.493622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:52.494815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:52.502651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1042/2064] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:52.506707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.516289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cce08-25e5-49ff-be2e-087342045a34 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.521352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.521940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.522213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.522480 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.523123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:26:52.523418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.525031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.537233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1023/2065] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:26:52.541460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.550683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931ddbc-eed7-4d86-84b4-7edbaa807ee5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.555357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.555985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.556269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.556546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.557188 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.559084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.587453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1043/2066] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:26:52 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:52.591670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.602457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c726f-7fcc-48d3-9cc9-9fac83271a3a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.605998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:52.675500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.676003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.679684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.679684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.679684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.679684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.700737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1024/2067] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:26:52 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:26:52.704880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.715500 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa16c6-5cb9-418e-a210-0f3c20e54fde admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.720210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.720804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.721083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.721816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.722585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.724212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.734059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1044/2068] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:26:52.738373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.748450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c24d6-f99e-4d73-a27e-1e30b7f58643 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.753751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.754336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.754620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.754900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.755623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.757227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4dc16-2e42-4bc6-8aff-20fff9ef4b9d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.767822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1025/2069] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:26:52.772091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.788050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab46b04-cd89-48d8-91ac-c21deb5265b9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.795321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.796025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.796312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.796590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] PATH_INFO: `/v3/projects/5472cdfe59524ac5b826fba54758c68f/users/0d773c7f6aca4ea2ad2872f67e9116af/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.813522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5472cdfe59524ac5b826fba54758c68f, user_id=0d773c7f6aca4ea2ad2872f67e9116af, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:52.816690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:52.829258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1045/2070] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:26:52 2022] PUT /identity/v3/projects/5472cdfe59524ac5b826fba54758c68f/users/0d773c7f6aca4ea2ad2872f67e9116af/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:26:52.834309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac38a-0c4f-42b0-9759-d72b4ebb875c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.834670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac38a-0c4f-42b0-9759-d72b4ebb875c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.835037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac38a-0c4f-42b0-9759-d72b4ebb875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:52.855971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c73ac38a-0c4f-42b0-9759-d72b4ebb875c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:52.857121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c73ac38a-0c4f-42b0-9759-d72b4ebb875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:52.918441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1026/2071] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:52.928568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:52.930391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:52.978052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:52.992806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aaa08-445a-4dcb-9aba-16119ca7e93f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:52.996504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:52.997109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:52.997387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:52.997658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:53.006919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:53.008404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d36c6b-e33a-4a50-a2af-aeec02b6413b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:53.017783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1046/2072] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:57.048029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:57.048661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:57.048661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:57.077821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] MFA Rules not processed for user `0d773c7f6aca4ea2ad2872f67e9116af`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:57.078964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:57.156734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1027/2073] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:57.167318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bf7d-dc90-4203-b1a8-75fb9a234272 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:57.167935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bf7d-dc90-4203-b1a8-75fb9a234272 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:57.167935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bf7d-dc90-4203-b1a8-75fb9a234272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:57.186934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3916bf7d-dc90-4203-b1a8-75fb9a234272 None None] MFA Rules not processed for user `0d773c7f6aca4ea2ad2872f67e9116af`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:57.188282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3916bf7d-dc90-4203-b1a8-75fb9a234272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:57.196435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1047/2074] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:26:57.208338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:26:57.209822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:57.228100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:26:57.287058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b8f8-db14-4a69-ba17-7003930d2df9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:26:57.292714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:26:57.293330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:57.293614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:57.293891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:57.303727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:26:57.306683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:26:57.316925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1028/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:26:58.215514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922c78-db76-439d-9f47-c64a7a172c4b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:26:58.216088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922c78-db76-439d-9f47-c64a7a172c4b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:26:58.216088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922c78-db76-439d-9f47-c64a7a172c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:26:58.262010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c5922c78-db76-439d-9f47-c64a7a172c4b None None] MFA Rules not processed for user `0d773c7f6aca4ea2ad2872f67e9116af`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:26:58.267262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c5922c78-db76-439d-9f47-c64a7a172c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:26:58.280897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1048/2076] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:00.943522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:00.951916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:00.995105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3290e0-18dc-4c2d-b082-9e30a9be833e service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:01.006441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:01.007822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:01.008310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:01.008832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] PATH_INFO: `/v3/users/d5476aacdb0b4015827a67be07e744ce` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:01.010396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5476aacdb0b4015827a67be07e744ce)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:01.013707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:01.129222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5476aacdb0b4015827a67be07e744ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.130443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5476aacdb0b4015827a67be07e744ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.148591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5476aacdb0b4015827a67be07e744ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.176358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5476aacdb0b4015827a67be07e744ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.177106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5476aacdb0b4015827a67be07e744ce'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.234852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1029/2077] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:00 2022] DELETE /identity/v3/users/d5476aacdb0b4015827a67be07e744ce => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:01.245889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:01.247178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:01.248843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:01.295019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:01.296630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:01.477233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1049/2078] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:01.588091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:01.591740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:01.753912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0cae1-f593-48a8-a3c6-cbd1371702ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:01.763841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:01.764836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:01.766261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:01.766766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:01.801737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:01.805732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415959e-361c-4540-aa49-d33590373f82 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:01.834769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1030/2079] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:01.872742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:01.874911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:01.890547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831756-a0b4-4521-93e5-708c328d03a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:01.896886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:01.897760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:01.898350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:01.898768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] PATH_INFO: `/v3/users/99be5b94102c40a683c86962db400249` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:01.900821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99be5b94102c40a683c86962db400249)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:01.903320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:01.970482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99be5b94102c40a683c86962db400249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.971196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99be5b94102c40a683c86962db400249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.981492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99be5b94102c40a683c86962db400249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.996969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99be5b94102c40a683c86962db400249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:01.998508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99be5b94102c40a683c86962db400249'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.016810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1050/2080] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:01 2022] DELETE /identity/v3/users/99be5b94102c40a683c86962db400249 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:02.021266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:02.021794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:02.021794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:02.042056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:02.044257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:02.059011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:02.148951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1031/2081] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:02.160801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:02.162436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:02.253901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:02.281233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454363f1-def8-4752-a416-fc3a7c5dcfb3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:02.290524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:02.291793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:02.293344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:02.293634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:02.309017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:02.311467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:02.325936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1051/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:02.691435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:02.693901 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:02.713448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023d708-5a2d-4812-926d-5d06887a06e2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:02.720642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:02.721932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:02.722421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:02.722842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] PATH_INFO: `/v3/projects/fec5fe8dd1c2442c8e16600d58389f30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:02.724286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec5fe8dd1c2442c8e16600d58389f30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:02.726923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:02.817287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec5fe8dd1c2442c8e16600d58389f30', 'request_id': 'req-e554279a-e35a-4aef-87cf-2153fa3ac030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.833747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec5fe8dd1c2442c8e16600d58389f30', 'request_id': 'req-e554279a-e35a-4aef-87cf-2153fa3ac030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.841141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec5fe8dd1c2442c8e16600d58389f30', 'request_id': 'req-e554279a-e35a-4aef-87cf-2153fa3ac030'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.853471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] The token cache is being invalidate because project fec5fe8dd1c2442c8e16600d58389f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:02.853772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.854625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:02.859584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1032/2083] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:02 2022] DELETE /identity/v3/projects/fec5fe8dd1c2442c8e16600d58389f30 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:02.860872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:02.879650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c9658-b37b-4d5b-8318-c78fd7fbc522 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:02.925403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:02.926869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:03.090403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:03.091495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:03.091495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:03.096916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] PATH_INFO: `/v3/users/f2412d4e49304717b549af50c2ca43c6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:03.101607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2412d4e49304717b549af50c2ca43c6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:03.107699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:03.138991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554279a-e35a-4aef-87cf-2153fa3ac030 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:03.157253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:03.158100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:03.158386 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:03.159672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] PATH_INFO: `/v3/projects/839400f909dc4caebb4ff9cfabed0d7c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:03.166533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839400f909dc4caebb4ff9cfabed0d7c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:03.169363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:03.258001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2412d4e49304717b549af50c2ca43c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.259730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2412d4e49304717b549af50c2ca43c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.293389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2412d4e49304717b549af50c2ca43c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.295997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:03.318836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2412d4e49304717b549af50c2ca43c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.319906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2412d4e49304717b549af50c2ca43c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.321692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839400f909dc4caebb4ff9cfabed0d7c', 'request_id': 'req-136f217a-8d1b-40c1-8aef-05eb993701a4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.336007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839400f909dc4caebb4ff9cfabed0d7c', 'request_id': 'req-136f217a-8d1b-40c1-8aef-05eb993701a4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.350155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839400f909dc4caebb4ff9cfabed0d7c', 'request_id': 'req-136f217a-8d1b-40c1-8aef-05eb993701a4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.368357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1052/2084] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:02 2022] DELETE /identity/v3/users/f2412d4e49304717b549af50c2ca43c6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:03.369628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:03.372671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:03.381077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] The token cache is being invalidate because project 839400f909dc4caebb4ff9cfabed0d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:03.382148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.388596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.391169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1033/2085] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:02 2022] DELETE /identity/v3/projects/839400f909dc4caebb4ff9cfabed0d7c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:03.396200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:03.400275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:03.560375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f217a-8d1b-40c1-8aef-05eb993701a4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:03.569019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:03.570405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:03.570937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:03.571573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] PATH_INFO: `/v3/users/232b41cbb1d44607b06f865239b392cd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:03.573364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=232b41cbb1d44607b06f865239b392cd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:03.575816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:03.580358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d48013-6cbf-4509-8bba-359217b2184f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:03.596830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:03.597879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:03.598335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:03.598773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] PATH_INFO: `/v3/users/c822832b45ce4825bc547e5a4c32bdc3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:03.600354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c822832b45ce4825bc547e5a4c32bdc3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:03.606245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:03.621533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:03.681058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232b41cbb1d44607b06f865239b392cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.682375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232b41cbb1d44607b06f865239b392cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.704612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232b41cbb1d44607b06f865239b392cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.746851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232b41cbb1d44607b06f865239b392cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.747887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-44ff0fc3-2d8a-4c62-9515-ef659b8a3704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232b41cbb1d44607b06f865239b392cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.751946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c822832b45ce4825bc547e5a4c32bdc3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.756496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c822832b45ce4825bc547e5a4c32bdc3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.787183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c822832b45ce4825bc547e5a4c32bdc3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.790309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1034/2086] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:03 2022] DELETE /identity/v3/users/232b41cbb1d44607b06f865239b392cd => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:03.818339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c822832b45ce4825bc547e5a4c32bdc3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.819298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c822832b45ce4825bc547e5a4c32bdc3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:03.854791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1053/2087] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:03 2022] DELETE /identity/v3/users/c822832b45ce4825bc547e5a4c32bdc3 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:03.859351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:03.859808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:03.860087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:03.905092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:03.906784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:03.912986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:03.914677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.035331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:04.088644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1035/2088] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:04.126654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.129397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.150690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d1b67-a385-4c72-9627-320e86b67f8b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.163996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.164850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.168118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.168576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] PATH_INFO: `/v3/projects/d7a8c07f648947a48554d464480ec7b8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.170658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a8c07f648947a48554d464480ec7b8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.173737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.238477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7626ef5-f311-4b5a-bc6b-74f3b379ebd1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.242950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.243587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.243875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.244394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.259941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.267579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.287774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a8c07f648947a48554d464480ec7b8', 'request_id': 'req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.300309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1036/2089] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:04.301459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a8c07f648947a48554d464480ec7b8', 'request_id': 'req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.304798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.306369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.311824 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a8c07f648947a48554d464480ec7b8', 'request_id': 'req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.335237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] The token cache is being invalidate because project d7a8c07f648947a48554d464480ec7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:04.335726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.337399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146f66cd-9b86-48b4-8e92-c4d9e1f52cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.337642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71059cb-ec65-47c5-90a8-4cab4ca58ae3 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.339306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1054/2090] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:03 2022] DELETE /identity/v3/projects/d7a8c07f648947a48554d464480ec7b8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:04.347372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d6eef-d5e9-4bf7-90d7-36b77fb0dfd4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.348246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d6eef-d5e9-4bf7-90d7-36b77fb0dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.348960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d6eef-d5e9-4bf7-90d7-36b77fb0dfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.382474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-078d6eef-d5e9-4bf7-90d7-36b77fb0dfd4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:04.384072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-078d6eef-d5e9-4bf7-90d7-36b77fb0dfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.402441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:04.526556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.526556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.526556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.526556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] PATH_INFO: `/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.537113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] RBAC: Authorizing `identity:delete_project(project_id=69600bf9ab0d46dd9dce1d6a6974480b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.542243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.567761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1055/2091] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:04.569825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.570450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.570827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.611034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:04.612881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.637571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1056/2092] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:04.638988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.650198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61a343-05ca-4cae-b2a3-c4577d004a6a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.653559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69600bf9ab0d46dd9dce1d6a6974480b', 'request_id': 'req-0d331d5a-ce24-4599-8824-29fb6679918a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.665553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69600bf9ab0d46dd9dce1d6a6974480b', 'request_id': 'req-0d331d5a-ce24-4599-8824-29fb6679918a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.676864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69600bf9ab0d46dd9dce1d6a6974480b', 'request_id': 'req-0d331d5a-ce24-4599-8824-29fb6679918a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.693122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] The token cache is being invalidate because project 69600bf9ab0d46dd9dce1d6a6974480b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:04.693583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.694632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.703581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1037/2093] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:04 2022] DELETE /identity/v3/projects/69600bf9ab0d46dd9dce1d6a6974480b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:04.749764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.751577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.751819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.752099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.752830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:04.753141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.755815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.792125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1057/2094] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:04.796641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.798558 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.806166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.810545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:04.819368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d331d5a-ce24-4599-8824-29fb6679918a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.826558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cdec-fa01-43d4-81c6-c8911073eed2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:04.827026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.827947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.828506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.829093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.830274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.833426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:04.834096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.834380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.835257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] PATH_INFO: `/v3/projects/9231500f143c462aaeed8c48a06d1760` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:04.836847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9231500f143c462aaeed8c48a06d1760)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:04.837005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.839324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:04.889960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:04.914618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1038/2095] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:04 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:04.918566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:04.934535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9231500f143c462aaeed8c48a06d1760', 'request_id': 'req-dbedc23e-9962-4a02-a685-c4fae1c0cceb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.947591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9231500f143c462aaeed8c48a06d1760', 'request_id': 'req-dbedc23e-9962-4a02-a685-c4fae1c0cceb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.958982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9231500f143c462aaeed8c48a06d1760', 'request_id': 'req-dbedc23e-9962-4a02-a685-c4fae1c0cceb'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.973516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] The token cache is being invalidate because project 9231500f143c462aaeed8c48a06d1760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:04.973951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.974647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dbedc23e-9962-4a02-a685-c4fae1c0cceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:04.976419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1058/2096] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:04 2022] DELETE /identity/v3/projects/9231500f143c462aaeed8c48a06d1760 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:04.982207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1299cb-51da-449f-80e5-cd3468df6e3b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:04.982501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1299cb-51da-449f-80e5-cd3468df6e3b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:04.982909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1299cb-51da-449f-80e5-cd3468df6e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.003889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d190f-2048-4454-8a69-5a7711df42c0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.013810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc1299cb-51da-449f-80e5-cd3468df6e3b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:05.015042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc1299cb-51da-449f-80e5-cd3468df6e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:05.042753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:05.072737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.073140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.073336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.073771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.074724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.076592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.101803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1039/2097] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:04 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:05.108277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1059/2098] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:05.108401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.108401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:05.110353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.110618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.110961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.122400 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a190984-497b-4c48-9879-2aa8f04abcb1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.129356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.130191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.130632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.130897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.131739 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.133259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.138365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:05.139618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:05.147735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1040/2099] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:05.148771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1060/2100] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:05.151974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.152878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.163634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4d509-9d45-4db8-a177-9a7842c530cf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.166803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c713a7-3d8e-4e5b-8570-c8fa47f02374 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.169567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.170855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.171141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.171416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.172616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:05.172953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.175706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.176930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.177556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.177838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.178303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.178964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.182426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.193727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1041/2101] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:05.198371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.203633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1061/2102] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:05.207879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.211192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c6185-548a-412e-9689-3e7341a48349 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.217831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.218506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.218783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.219055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] PATH_INFO: `/v3/projects/a1d3e76c24984edb82acfa9960c08710/users/98e50bf9dc8749698227fe8dd2c0c9bc/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.220495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098803-92b1-4bbb-8289-7b28e388fce2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.230058 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.231086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.231458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.232459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.234378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.237906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.247818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d3e76c24984edb82acfa9960c08710, user_id=98e50bf9dc8749698227fe8dd2c0c9bc, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.251289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c386bdd-b1bf-416b-9c0a-67e06c848c0f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.279583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1042/2103] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:05 2022] PUT /identity/v3/projects/a1d3e76c24984edb82acfa9960c08710/users/98e50bf9dc8749698227fe8dd2c0c9bc/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:05.286746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.287209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.287525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.295836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1062/2104] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:05.303557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.326389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] MFA Rules not processed for user `98e50bf9dc8749698227fe8dd2c0c9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:05.327723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:05.378051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a5791-4de3-468e-86b6-67c9e8b04900 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.384056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.384968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.385379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.385852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.386690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.388780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.391347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:05.416074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1063/2105] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:05.420177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.427843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1043/2106] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:05.431161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.435599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecffbd-cf3c-418b-82c2-8a24b4592c13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.440371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.441154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.441498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.442144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.442568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.443414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155cb65-9e03-4ee3-a4b7-718eafc16e64 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.444365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.448377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.449519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.449519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.449708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.450619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.452615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.459224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1064/2107] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:05.461179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.477951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0186d6-cada-4535-9985-9d1dd364aa01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.484066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.484818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.485238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.485984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.486981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.489001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.489276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1044/2108] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:05.494276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.506175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1065/2109] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:05.510302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.527156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d090-d27e-4f0f-83a0-2cd9007bb4e9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.613164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:05.619937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eeca4-cf11-4e5f-a6c5-17db0299fc8f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.622588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.623278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.623598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.623850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] PATH_INFO: `/v3/projects/91c51d3fd1114acbac4bae6c8eaf1a6e/users/6b09dcdc43be4c57b75f5f4f784758fb/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.646291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.646930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.647212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.647485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.648166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.650093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.657868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c51d3fd1114acbac4bae6c8eaf1a6e, user_id=6b09dcdc43be4c57b75f5f4f784758fb, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.663087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6a488-c404-4725-9075-625744e1bc85 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.686595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1045/2110] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:05.690506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.700557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1066/2111] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:05 2022] PUT /identity/v3/projects/91c51d3fd1114acbac4bae6c8eaf1a6e/users/6b09dcdc43be4c57b75f5f4f784758fb/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:05.701704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb274bd-ab86-477f-bdf6-38650e4177cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.703313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306525-6299-496f-9247-34b66079b0d3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.707120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306525-6299-496f-9247-34b66079b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.708106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306525-6299-496f-9247-34b66079b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.748725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-63306525-6299-496f-9247-34b66079b0d3 None None] MFA Rules not processed for user `6b09dcdc43be4c57b75f5f4f784758fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:05.750116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-63306525-6299-496f-9247-34b66079b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:05.809838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.810771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.811569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.811569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.812517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.813918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.827524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1046/2112] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:05.829181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.840216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077a22e-e4ca-4e20-9bb4-4e1d4df36c23 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.846434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.847378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.847696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.848338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] PATH_INFO: `/v3/projects/429a9727aa64496b85830fe8716094c9/users/8da33d1771c64942bc612003308d98f4/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.861023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1067/2113] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:05.865739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306525-6299-496f-9247-34b66079b0d3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.869662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=429a9727aa64496b85830fe8716094c9, user_id=8da33d1771c64942bc612003308d98f4, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.872979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.889241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306525-6299-496f-9247-34b66079b0d3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.889846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1047/2114] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:05 2022] PUT /identity/v3/projects/429a9727aa64496b85830fe8716094c9/users/8da33d1771c64942bc612003308d98f4/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:05.894392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:05.928310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:05.968462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.969083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.969519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.969798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.970775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.974448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:05.990115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189543d-86b7-4d91-b5ec-0c7ec13fa5db admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:05.995317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:05.996414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:05.996796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:05.997086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:05.997743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:05.999483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.014149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1048/2115] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:06.021298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.029947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1068/2116] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:05 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:06.035296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.037702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd8b91-a48b-4244-a85b-5142d1305f88 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.102119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:06.119838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.120691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.120830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b17b1-00b1-4681-9e58-baf9ceeff687 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.120974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.121248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] PATH_INFO: `/v3/projects/429a9727aa64496b85830fe8716094c9/users/8da33d1771c64942bc612003308d98f4/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.123176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=429a9727aa64496b85830fe8716094c9, user_id=8da33d1771c64942bc612003308d98f4, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.125672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.125875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83953560-c94d-4751-94f2-8ed917e74dae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.126271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.126549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.126819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.127783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.129721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.142513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1049/2117] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:06 2022] PUT /identity/v3/projects/429a9727aa64496b85830fe8716094c9/users/8da33d1771c64942bc612003308d98f4/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:06.154638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.154929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.155256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.163367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1069/2118] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:06 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:06.164431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.175588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4777d2-27a4-436b-9a92-f87359b0fea8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.190537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] MFA Rules not processed for user `8da33d1771c64942bc612003308d98f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:06.191815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:06.249564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.250349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.250722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.251137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.251896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.253833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.265581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1070/2119] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:06.269927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.297028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1050/2120] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:06.300863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce0f00-adde-4495-b06e-2fb849859be1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.308178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.309067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.309640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.310263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.311225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.311683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.313712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:06.313935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.331417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1071/2121] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:06.342334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.353262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab20b1-8f4e-4525-94a5-aabd9f48f1d8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.358808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.359614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.360039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.360460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] PATH_INFO: `/v3/projects/8ffaef2488ee46a8b82c9452424a48a9/users/2dd93cd417a3493a80a2e026f773e105/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.383015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:06.385958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffaef2488ee46a8b82c9452424a48a9, user_id=2dd93cd417a3493a80a2e026f773e105, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.390041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f665e6-47f1-4a84-b935-ba77eac3f03d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.405768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1c127-acbf-43b5-bf6c-87a051d984d3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.407732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1072/2122] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:06 2022] PUT /identity/v3/projects/8ffaef2488ee46a8b82c9452424a48a9/users/2dd93cd417a3493a80a2e026f773e105/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:06.418998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.419282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.419598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.459905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] MFA Rules not processed for user `2dd93cd417a3493a80a2e026f773e105`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:06.461407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:06.480014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.481453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.481832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.482107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.552494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1073/2123] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:06.557104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.559454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.560930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.572429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1051/2124] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:06.599225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:06.647023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5bb62-456d-4fc9-a398-e115ae799a1b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.653008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.653874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.654611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.654898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.655807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.659580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.696482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1074/2125] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:06 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:06.696680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.703172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.707563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6bf6-bdcd-4d31-8226-4eef8d60d6af service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.765079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.765696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.765979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.766260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.796730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89a4c-5aa5-48f9-9e2f-98fcc1e24f9c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.803790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.804351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.804764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.805036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.805663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.807598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.838026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1075/2126] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:06 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:06.841792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.856972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860c148-8c7a-4b01-b345-dea2863e564a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.865599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.866756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.867369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.867860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.869172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.870719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:06.874139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.876012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.876544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.892387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1052/2127] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:06.896917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1076/2128] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:06.899229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.919957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb74a3-b858-4378-9b59-124f865c7fba service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.926247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.926992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.927829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.927829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] PATH_INFO: `/v3/projects/29208945f4ca42e0a94fb2288f07de89/users/b982c2860d4b42ccbbdf4f7c4aa932f7/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.933541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:06.935363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:06.952884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a487d44-6f01-4665-aa01-10c362a39f6d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:06.955660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29208945f4ca42e0a94fb2288f07de89, user_id=b982c2860d4b42ccbbdf4f7c4aa932f7, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.958466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.963900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:06.964712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:06.965112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:06.965456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] PATH_INFO: `/v3/users/b1c18aadf10d46018ee874f47fb9b9d6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:06.966829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c18aadf10d46018ee874f47fb9b9d6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:06.969124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:06.983269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1053/2129] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:06 2022] PUT /identity/v3/projects/29208945f4ca42e0a94fb2288f07de89/users/b982c2860d4b42ccbbdf4f7c4aa932f7/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:06.984496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:07.089763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c18aadf10d46018ee874f47fb9b9d6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.092227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c18aadf10d46018ee874f47fb9b9d6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.093721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf2ff-90e9-4cb6-9cc2-99c74065c227 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:07.100069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:07.101008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.101642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.102021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.112290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:07.112290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:07.112619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c18aadf10d46018ee874f47fb9b9d6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.125303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1054/2130] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:07.126319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:07.128707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.134666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c18aadf10d46018ee874f47fb9b9d6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.141213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-baa97e70-fc6d-4a01-92ac-8efb0cdb1d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c18aadf10d46018ee874f47fb9b9d6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.144224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18acc1-f432-4eaa-911f-a0acc0961ee5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:07.151106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:07.152003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.152325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.152688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] PATH_INFO: `/v3/projects/29208945f4ca42e0a94fb2288f07de89/users/b982c2860d4b42ccbbdf4f7c4aa932f7/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.155056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29208945f4ca42e0a94fb2288f07de89, user_id=b982c2860d4b42ccbbdf4f7c4aa932f7, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:07.159203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:07.169413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1077/2131] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:06 2022] DELETE /identity/v3/users/b1c18aadf10d46018ee874f47fb9b9d6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:07.186062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1055/2132] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:07 2022] PUT /identity/v3/projects/29208945f4ca42e0a94fb2288f07de89/users/b982c2860d4b42ccbbdf4f7c4aa932f7/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:07.186457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.186910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.187333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.208580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:07.242522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] MFA Rules not processed for user `b982c2860d4b42ccbbdf4f7c4aa932f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:07.242522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.386790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1078/2133] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:07.431390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:07.434491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.471651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:07.473432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.531591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400f477-a5a2-4dc5-b916-900ed5d2261b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:07.560131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c0db-9e93-4ec9-b4d7-1d23e3bc75a2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:07.570028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:07.570831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.571245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.571548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] PATH_INFO: `/v3/projects/af0a3b65874744918c60a2cf098cdbac` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.572875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0a3b65874744918c60a2cf098cdbac)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:07.575735 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:07.591959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:07.592673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.593025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.593368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.595269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.628689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:07.652758 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0a3b65874744918c60a2cf098cdbac', 'request_id': 'req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.661769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0a3b65874744918c60a2cf098cdbac', 'request_id': 'req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.668437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0a3b65874744918c60a2cf098cdbac', 'request_id': 'req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.673237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:07.675988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:07.682897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] The token cache is being invalidate because project af0a3b65874744918c60a2cf098cdbac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:07.683194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.683914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.685672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1079/2134] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:07 2022] DELETE /identity/v3/projects/af0a3b65874744918c60a2cf098cdbac => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:07.689642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:07.691729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1056/2135] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:07.692290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:07.723079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:07.785364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52130-1c96-40e2-9f5e-bf26a86fcd61 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:07.790406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:07.791190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:07.791456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:07.791787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] PATH_INFO: `/v3/users/3c84c02d78624ad990a3d424273f6e11` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:07.793011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c84c02d78624ad990a3d424273f6e11)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:07.795011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:07.860639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c84c02d78624ad990a3d424273f6e11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.861569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c84c02d78624ad990a3d424273f6e11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.872285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c84c02d78624ad990a3d424273f6e11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.891086 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c84c02d78624ad990a3d424273f6e11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.892144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b9ff03b0-ea54-47c4-ac7c-6fca44c46fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c84c02d78624ad990a3d424273f6e11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:07.912958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1080/2136] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:07 2022] DELETE /identity/v3/users/3c84c02d78624ad990a3d424273f6e11 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:08.168396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.185976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65738fbe-28bd-4d47-87d6-8fe7d39b5e64 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.256305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.256947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.257229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.258190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] PATH_INFO: `/v3/projects/68f36d31a8bc4b59816263390a5bb46e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.259234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f36d31a8bc4b59816263390a5bb46e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.261263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.309321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f36d31a8bc4b59816263390a5bb46e', 'request_id': 'req-22f0eb4d-f033-4600-a698-0374b46d722b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:08.316767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f36d31a8bc4b59816263390a5bb46e', 'request_id': 'req-22f0eb4d-f033-4600-a698-0374b46d722b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:08.323367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f36d31a8bc4b59816263390a5bb46e', 'request_id': 'req-22f0eb4d-f033-4600-a698-0374b46d722b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:08.333860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] The token cache is being invalidate because project 68f36d31a8bc4b59816263390a5bb46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:08.334143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:08.334788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22f0eb4d-f033-4600-a698-0374b46d722b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:08.336367 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1057/2137] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:08 2022] DELETE /identity/v3/projects/68f36d31a8bc4b59816263390a5bb46e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:08.341132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.341425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.341703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.363353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:08.364519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:08.437497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1081/2138] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:08.445337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.445733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.445897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.470487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:08.473109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:08.487108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1058/2139] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:08.491672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.505021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a463e6-f54e-4835-8c36-5469b4925a35 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.510123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.510780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.511081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.511365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.512117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:08.512405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.515630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.531727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1082/2140] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:08.537077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.546521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33292a1-acc8-4aa6-8bb3-21d62a470bbd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.551784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.552388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.552669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.552941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.553587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.555799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.598771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1059/2141] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:08.600975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.612911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720656d-b3c6-48ab-84ac-fccbd3461a47 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.647526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:08.689072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.690581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.691026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.691512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.692676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.695710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.731570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1083/2142] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:08 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:08.736781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.750089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc872-b7a9-440f-b5f1-f8cf150b41ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.755584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.756197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.756484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.756761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.757411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.759017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.769208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1060/2143] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:08.773380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.783924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7bc3d-87dd-4ecf-ab1b-de5c890eb0bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.788839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.789433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.789814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.789972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.790613 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.792267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c4727-20ba-4dbf-b82e-4c17222c61a5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.805061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1084/2144] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:08.809667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.820103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de477c-689f-44bd-a8ed-72f7a6689554 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:08.824668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:08.825304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.825581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.825851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] PATH_INFO: `/v3/projects/587f20913f4840fdb2914563f7f0c135/users/1057d55d591a42f4b3ed848b62995861/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.841795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587f20913f4840fdb2914563f7f0c135, user_id=1057d55d591a42f4b3ed848b62995861, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:08.842229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:08.845468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:08.860972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1061/2145] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:08 2022] PUT /identity/v3/projects/587f20913f4840fdb2914563f7f0c135/users/1057d55d591a42f4b3ed848b62995861/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:08.872759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:08.872981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:08.873259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:08.903004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] MFA Rules not processed for user `1057d55d591a42f4b3ed848b62995861`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:08.904250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:08.971712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1085/2146] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:08.983057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:08.984831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.042759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a831-7bfc-41ce-80a7-e29dbeb9a406 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:09.046754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:09.047353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.047667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.047943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.057035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:09.058502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72ede-9a00-49e2-920e-2c18ec29927f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:09.069298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1062/2147] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:09.090770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:09.155851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefc914-05bb-488f-a1af-75e081a13472 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:09.162441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:09.163078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.163356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.163670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] PATH_INFO: `/v3/users/1057d55d591a42f4b3ed848b62995861` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.164817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1057d55d591a42f4b3ed848b62995861)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:09.168464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:09.211503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:09.223326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1057d55d591a42f4b3ed848b62995861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.224902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1057d55d591a42f4b3ed848b62995861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.232925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1057d55d591a42f4b3ed848b62995861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.245149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1057d55d591a42f4b3ed848b62995861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.245852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1057d55d591a42f4b3ed848b62995861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.257960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1086/2148] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:09 2022] DELETE /identity/v3/users/1057d55d591a42f4b3ed848b62995861 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:09.262398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.262709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.262964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.283098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:09.285263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.348671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1063/2149] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:09.358770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:09.360783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.432089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987f2bf-d15b-4b08-9149-1c969684fb92 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:09.439960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:09.440647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.441397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.441397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.455014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:09.456984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:09.468429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1087/2150] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:09.762875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:09.764445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.772934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:09.774443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.778884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d18e3-3c18-4951-ad2f-246b5474daea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:09.785152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:09.785794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.786110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.786387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] PATH_INFO: `/v3/users/98e50bf9dc8749698227fe8dd2c0c9bc` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.787524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e50bf9dc8749698227fe8dd2c0c9bc)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:09.789565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:09.800181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd2d7-4a81-4528-8c86-13de28dfe804 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:09.805390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:09.806018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.806297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.806567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] PATH_INFO: `/v3/projects/587f20913f4840fdb2914563f7f0c135` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.807722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=587f20913f4840fdb2914563f7f0c135)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:09.809701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:09.841668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e50bf9dc8749698227fe8dd2c0c9bc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.842314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e50bf9dc8749698227fe8dd2c0c9bc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.850102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e50bf9dc8749698227fe8dd2c0c9bc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.859527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587f20913f4840fdb2914563f7f0c135', 'request_id': 'req-5058f731-1043-4656-a1d1-9a9fbf750c6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.862396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e50bf9dc8749698227fe8dd2c0c9bc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.863248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5dd6a459-c4de-416d-ad45-c1e22a739dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e50bf9dc8749698227fe8dd2c0c9bc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.868265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587f20913f4840fdb2914563f7f0c135', 'request_id': 'req-5058f731-1043-4656-a1d1-9a9fbf750c6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.874594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587f20913f4840fdb2914563f7f0c135', 'request_id': 'req-5058f731-1043-4656-a1d1-9a9fbf750c6f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.876017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1064/2151] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:09 2022] DELETE /identity/v3/users/98e50bf9dc8749698227fe8dd2c0c9bc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:09.880658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.881027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.881304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.886461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] The token cache is being invalidate because project 587f20913f4840fdb2914563f7f0c135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:09.886690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.887338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5058f731-1043-4656-a1d1-9a9fbf750c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:09.889001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1088/2152] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:09 2022] DELETE /identity/v3/projects/587f20913f4840fdb2914563f7f0c135 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:09.893822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca02d3-bced-4490-b929-1322a71d5136 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.894106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca02d3-bced-4490-b929-1322a71d5136 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.894379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca02d3-bced-4490-b929-1322a71d5136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:09.895294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:09.902202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:09.903902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.915211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9bca02d3-bced-4490-b929-1322a71d5136 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:09.916428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9bca02d3-bced-4490-b929-1322a71d5136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.953935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:09.982810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1065/2153] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:09.987088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1089/2154] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:09.993016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:09.994464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:09.997003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:09.997293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:09.997569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.020583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:10.021869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.032583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1090/2155] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:10.037115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.048785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3290e-0981-42cf-854f-ea421258da7b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.062706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dd726-3bf4-4ea8-856e-8676c2837bb3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.064581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.065891 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.066241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.066521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.067184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:10.067514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.068754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.069140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.070091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.070538 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.071005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.083303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.084851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.091316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1091/2156] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:27:10.092396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.096601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1066/2157] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:10.103978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58f8ad-329a-49ef-bf22-a32bde383ead admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.109752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.110602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.110950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.111291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.112756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.115078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.147049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1092/2158] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:10.151236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.162749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374c724-52d6-4115-bf5c-bc8e424948e6 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.232765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.233793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.234084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.234605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.235253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.237759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.269926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1067/2159] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:10 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:10.274403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.286490 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b612327-3f99-4e2e-bb1d-4ecc5c6ffbaa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.295699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.295699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.295699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.295699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.296026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.296933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.311247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1093/2160] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:10.312323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.324718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f2c6e-e866-4233-bc9b-abef401305cb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.329897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.330791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.331156 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.331336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.332023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.333616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.344033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1068/2161] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:10.354408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.370801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e1c0-e30e-4763-9339-1b4130d0f157 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.376712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.377409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.377771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.383581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] PATH_INFO: `/v3/projects/bdf0d6145a4848a8bc624ded77bc27ac/users/ba45eee6687b45158555ef5f0c595680/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.398554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf0d6145a4848a8bc624ded77bc27ac, user_id=ba45eee6687b45158555ef5f0c595680, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.401528 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.418615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.420247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1094/2162] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:10 2022] PUT /identity/v3/projects/bdf0d6145a4848a8bc624ded77bc27ac/users/ba45eee6687b45158555ef5f0c595680/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:10.421205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.425577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.427374 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.447287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c25f5-aa06-457e-9d2d-76967074aa7a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.479651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:10.494414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29d89-1c8e-4e44-992a-60de869c3595 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.498186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.498870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.499151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.499427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.501280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.515180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.515839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.516120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.516389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] PATH_INFO: `/v3/projects/a1d3e76c24984edb82acfa9960c08710` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.517511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d3e76c24984edb82acfa9960c08710)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.520027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.562702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.564202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc472c-9447-4908-9396-c890c64f97f7 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.568659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d3e76c24984edb82acfa9960c08710', 'request_id': 'req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.573453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1095/2163] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:10.574685 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e570dc-9873-4d06-a195-261289b12670 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.574966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e570dc-9873-4d06-a195-261289b12670 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.575284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e570dc-9873-4d06-a195-261289b12670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.585972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d3e76c24984edb82acfa9960c08710', 'request_id': 'req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.594285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d3e76c24984edb82acfa9960c08710', 'request_id': 'req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.603828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-09e570dc-9873-4d06-a195-261289b12670 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:10.605465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09e570dc-9873-4d06-a195-261289b12670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.606444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] The token cache is being invalidate because project a1d3e76c24984edb82acfa9960c08710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:10.606675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.607821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.613171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1069/2164] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:10 2022] DELETE /identity/v3/projects/a1d3e76c24984edb82acfa9960c08710 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:10.614648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.616500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.622796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:10.720863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdcf13-40d3-4a77-8a54-0a7c0b1bad21 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.727337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.727766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1096/2165] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:10.728251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.728569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.729450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] PATH_INFO: `/v3/users/8da33d1771c64942bc612003308d98f4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.730167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da33d1771c64942bc612003308d98f4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.732280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e570dc-9873-4d06-a195-261289b12670 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:10.733381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.737457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09e570dc-9873-4d06-a195-261289b12670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.803420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da33d1771c64942bc612003308d98f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.804549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da33d1771c64942bc612003308d98f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.818446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da33d1771c64942bc612003308d98f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.821247 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e570dc-9873-4d06-a195-261289b12670 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:10.824012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:10.825084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:10.825810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:10.826038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:10.826346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:10.828199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:10.843917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da33d1771c64942bc612003308d98f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.843917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da33d1771c64942bc612003308d98f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:10.844261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:10.846313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:10.858066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1097/2166] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:10.858846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1070/2167] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:10 2022] DELETE /identity/v3/users/8da33d1771c64942bc612003308d98f4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:11.219299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.222174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.258556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a8e7c-d4e5-49b5-848f-e55b9ea4f5fa service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.325119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.326850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.343833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:11.344599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.344914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.345206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] PATH_INFO: `/v3/projects/429a9727aa64496b85830fe8716094c9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.348424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429a9727aa64496b85830fe8716094c9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:11.353416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:11.412285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03401bb7-339f-48a8-bdec-a3f8862f783a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.424798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429a9727aa64496b85830fe8716094c9', 'request_id': 'req-f4622eef-6c38-43ee-bb48-761d595f5c11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:11.438319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429a9727aa64496b85830fe8716094c9', 'request_id': 'req-f4622eef-6c38-43ee-bb48-761d595f5c11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:11.444994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:11.446369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429a9727aa64496b85830fe8716094c9', 'request_id': 'req-f4622eef-6c38-43ee-bb48-761d595f5c11'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:11.467001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] The token cache is being invalidate because project 429a9727aa64496b85830fe8716094c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:11.467608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:11.468693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f4622eef-6c38-43ee-bb48-761d595f5c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:11.474795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1098/2168] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:11 2022] DELETE /identity/v3/projects/429a9727aa64496b85830fe8716094c9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:11.481002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa1ce-3905-482c-9db3-1b2d298377c6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.481297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa1ce-3905-482c-9db3-1b2d298377c6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.481992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaa1ce-3905-482c-9db3-1b2d298377c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.507855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4dbaa1ce-3905-482c-9db3-1b2d298377c6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:11.509389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4dbaa1ce-3905-482c-9db3-1b2d298377c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.525655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:11.526282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.526673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.526842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.529206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.655965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1099/2169] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:11.661845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597594-4485-4640-8b54-12d8924916ec None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.662248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597594-4485-4640-8b54-12d8924916ec None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.662596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597594-4485-4640-8b54-12d8924916ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.678048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:11.679703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aca2d-671e-4f76-a316-a23cad66c339 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:11.689445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b6597594-4485-4640-8b54-12d8924916ec None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:11.691262 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b6597594-4485-4640-8b54-12d8924916ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.699922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1071/2170] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:11.701509 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.702832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.706461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.706573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1100/2171] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:11.707624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6597594-4485-4640-8b54-12d8924916ec None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.732884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6597594-4485-4640-8b54-12d8924916ec None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.748868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:11.748868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.749182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.749446 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.750080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:11.750415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:11.752783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:11.766659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] MFA Rules not processed for user `ba45eee6687b45158555ef5f0c595680`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:11.771577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1101/2172] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:11.771742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.773934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.790493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eac2aa-61e5-4ee6-857c-94d1ef33931d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.798803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:11.799916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.801173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.801902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.802928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:11.805408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:11.809444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:11.861115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1102/2173] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:11 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:11.865495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.887925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1072/2174] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:11.906774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:11.909966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:11.927393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f3c84-cbf7-416c-bc2d-6b5ef3831c20 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.965312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7ee3-acbd-44bd-98fb-6ecc2067860a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:11.972475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:11.973472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:11.973798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:11.974087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:11.974799 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:11.976838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:11.999320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.000096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.000381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.000657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.009961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1103/2175] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:11 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:12.013537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.025661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c5f7c-c43d-4773-970d-c1deec25a3e1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.031173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.031872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.032155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.032430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.033605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.035709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.047577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1104/2176] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:12.061296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.073875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb86b15-8831-4819-8665-53bdcc9ae450 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.079211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.081675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.085650 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.086487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.086763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.087113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.087879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.089615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696063d-e823-45a0-a15a-a2b2e4782ddf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.095581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1073/2177] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:12.102553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1105/2178] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:12.106645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.119852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012e5ac-c745-4dfa-9745-a85ac25ed67f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.124942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.125637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.125919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.126193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] PATH_INFO: `/v3/projects/518475493492462693b476f8dcd125d3/users/2b53b0c227bf4fdaaf20712e68bf42ff/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.146890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518475493492462693b476f8dcd125d3, user_id=2b53b0c227bf4fdaaf20712e68bf42ff, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.149586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.164230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1074/2179] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:12 2022] PUT /identity/v3/projects/518475493492462693b476f8dcd125d3/users/2b53b0c227bf4fdaaf20712e68bf42ff/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:12.175936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.176223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.176555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.206393 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] MFA Rules not processed for user `2b53b0c227bf4fdaaf20712e68bf42ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:12.207772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:12.213156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:12.279220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1106/2180] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:12.285127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.353262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675bb41-6cfb-4db4-8ba8-713d7d602ec2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.358223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.370663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.370663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.370663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.370663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.370663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.374211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:12.395349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1075/2181] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:12.400699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.418450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f58e1-909e-40fc-a5ba-44b9e3268080 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.484550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.485157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.485440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.485716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.487008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.489003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.510259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1107/2182] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:12.514354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.525033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc9743-a455-41dd-86de-5829b23225f0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.529735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.530336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.530626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.530899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.531554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.533703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.544492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1076/2183] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:12.548493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.559233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfc84d-b69f-4cae-90b0-73c7359558f7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.564131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.564935 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.565208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.565477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.566110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.567721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09add507-f641-4877-9fef-5ba8fc420b21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.577634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1108/2184] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:12.581754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.591749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1efb45-8e9f-40da-a69e-3fdf3f1a4749 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.597148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.597784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.598062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.598331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] PATH_INFO: `/v3/projects/f2fbcd12a5744ae9bf6494bc7c673aee/users/a1ab3c5722b54e0d87aa8d78e481252a/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.613462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fbcd12a5744ae9bf6494bc7c673aee, user_id=a1ab3c5722b54e0d87aa8d78e481252a, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.616137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.629057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1077/2185] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:12 2022] PUT /identity/v3/projects/f2fbcd12a5744ae9bf6494bc7c673aee/users/a1ab3c5722b54e0d87aa8d78e481252a/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:12.639179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.639475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.639789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.668388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] MFA Rules not processed for user `a1ab3c5722b54e0d87aa8d78e481252a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:12.669600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:12.733704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1109/2186] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:12.738547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.804120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fda0f-0dad-4c05-a7c6-6d702d563c84 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.808925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.809568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.810125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.810287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.810929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.812878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.842609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1078/2187] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:12.843937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.855722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd2527-59ff-4291-b509-01807dd2b74f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.867104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:12.914358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.915006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.915289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.915600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.916245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.918166 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.950208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1110/2188] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:12 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:12.951515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.962876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc2f48-60cc-40a0-b4ab-24b8ff6ba5f2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:12.967734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:12.968399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:12.968753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:12.969212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:12.969872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:12.972166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:12.982812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1079/2189] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:12.987095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:12.997111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24685e8-3cef-45d8-ba78-9ebe33e7424f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:13.001877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:13.002615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.002972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.003316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] PATH_INFO: `/v3/projects/430981af748c4753acbda270141d8c28/users/3902d8b666ce4414aea41538796d09f9/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.019333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430981af748c4753acbda270141d8c28, user_id=3902d8b666ce4414aea41538796d09f9, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:13.022027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:13.036025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1111/2190] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:12 2022] PUT /identity/v3/projects/430981af748c4753acbda270141d8c28/users/3902d8b666ce4414aea41538796d09f9/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:13.039941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:13.049761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d78b-e214-4479-9cb9-467730d37ede admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:13.110656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:13.112176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.112611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.112878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.113518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:13.115240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861588d-8b6c-4d86-97ca-5c3b9bf10971 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:13.126795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1080/2191] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:13.131261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:13.166495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd406ec-680e-4847-945a-14eec4e6dd24 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:13.172414 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:13.173271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.173745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.174737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] PATH_INFO: `/v3/projects/430981af748c4753acbda270141d8c28/users/3902d8b666ce4414aea41538796d09f9/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.177425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430981af748c4753acbda270141d8c28, user_id=3902d8b666ce4414aea41538796d09f9, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:13.180798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:13.199525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1112/2192] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:13 2022] PUT /identity/v3/projects/430981af748c4753acbda270141d8c28/users/3902d8b666ce4414aea41538796d09f9/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:13.211695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.212145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.212516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.256106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] MFA Rules not processed for user `3902d8b666ce4414aea41538796d09f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:13.256106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:13.267324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:13.349871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1081/2193] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:13.362320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:13.422012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa50d7-6edc-4b8a-a521-2aa08cf131d6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:13.426507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:13.427123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.427420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.427716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.449015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:13.450576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4aff0-5e87-4be3-8f23-c4d2cd9eb99c service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:13.463245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1113/2194] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:13.514369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:13.515871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:13.528521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee5d7d-09d0-448e-a616-c17e40dee191 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:13.532504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:13.533977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:13.534523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:13.535019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:13.598704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:13.600200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:13.612469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1082/2195] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:14.549655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09f00-c1c8-42ab-894a-387c353c505a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:14.550152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09f00-c1c8-42ab-894a-387c353c505a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:14.550238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09f00-c1c8-42ab-894a-387c353c505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:14.569471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-39a09f00-c1c8-42ab-894a-387c353c505a None None] MFA Rules not processed for user `ba45eee6687b45158555ef5f0c595680`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:14.570602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39a09f00-c1c8-42ab-894a-387c353c505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:14.624046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1114/2196] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:16.951382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:16.952817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.044415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe40e48-3fd7-4c12-a453-b37ae053b646 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.048891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.049599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.049873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.050143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] PATH_INFO: `/v3/users/6b09dcdc43be4c57b75f5f4f784758fb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.051267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b09dcdc43be4c57b75f5f4f784758fb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.053202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.103208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b09dcdc43be4c57b75f5f4f784758fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.103892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b09dcdc43be4c57b75f5f4f784758fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.106055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:17.113154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b09dcdc43be4c57b75f5f4f784758fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.124848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b09dcdc43be4c57b75f5f4f784758fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.125537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b09dcdc43be4c57b75f5f4f784758fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.138604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1083/2197] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:16 2022] DELETE /identity/v3/users/6b09dcdc43be4c57b75f5f4f784758fb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:17.143732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.144207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.144301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.164030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:17.165227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.176184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:17.229328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1115/2198] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:17.239205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:17.240633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.299562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a34fb-77ce-4b93-8719-6fed777e128f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.303142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.303782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.304064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.304335 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.315501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.316999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.326564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1084/2199] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:17.594179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:17.595797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.607144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:17.608584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.624903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e86763-b1aa-46cb-9391-3e5ac9bb0ff3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.648136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecada890-63f9-4a0c-879a-df64132ac175 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.654544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.655181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.655460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.655763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] PATH_INFO: `/v3/projects/91c51d3fd1114acbac4bae6c8eaf1a6e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.656894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c51d3fd1114acbac4bae6c8eaf1a6e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.659713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.660292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.660948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.661458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.661772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] PATH_INFO: `/v3/users/2b53b0c227bf4fdaaf20712e68bf42ff` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.663518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b53b0c227bf4fdaaf20712e68bf42ff)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.665825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.726712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c51d3fd1114acbac4bae6c8eaf1a6e', 'request_id': 'req-b718a01f-2267-45c8-8007-2cb3b845574e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.738341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c51d3fd1114acbac4bae6c8eaf1a6e', 'request_id': 'req-b718a01f-2267-45c8-8007-2cb3b845574e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.738540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b53b0c227bf4fdaaf20712e68bf42ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.739565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b53b0c227bf4fdaaf20712e68bf42ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.747339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b53b0c227bf4fdaaf20712e68bf42ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.748989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c51d3fd1114acbac4bae6c8eaf1a6e', 'request_id': 'req-b718a01f-2267-45c8-8007-2cb3b845574e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.764155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] The token cache is being invalidate because project 91c51d3fd1114acbac4bae6c8eaf1a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:17.764461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.765186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.771525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1085/2200] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:17 2022] DELETE /identity/v3/projects/91c51d3fd1114acbac4bae6c8eaf1a6e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:17.772220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b53b0c227bf4fdaaf20712e68bf42ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.772425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:17.773327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b71f1fcb-80a4-46bc-b7f6-a8c3c78d406e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b53b0c227bf4fdaaf20712e68bf42ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.774053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.793245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1116/2201] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:17 2022] DELETE /identity/v3/users/2b53b0c227bf4fdaaf20712e68bf42ff => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:17.798070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.799724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.800067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.821349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:17.822572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.833700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:17.848515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718a01f-2267-45c8-8007-2cb3b845574e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.884155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1117/2202] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:17.895681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:17.897077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:17.915033 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.915977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.916258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.916562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] PATH_INFO: `/v3/users/2dd93cd417a3493a80a2e026f773e105` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.917703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd93cd417a3493a80a2e026f773e105)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.919809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.929279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:17.960868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80785300-bb47-4aa7-8069-16daa3576d53 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:17.964542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:17.965141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:17.965418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:17.965687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:17.972711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd93cd417a3493a80a2e026f773e105'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.973635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd93cd417a3493a80a2e026f773e105'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.975252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:17.976764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:17.980877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd93cd417a3493a80a2e026f773e105'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.986997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1118/2203] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:17.995502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd93cd417a3493a80a2e026f773e105'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:17.996358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd93cd417a3493a80a2e026f773e105'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.012952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1086/2204] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:17 2022] DELETE /identity/v3/users/2dd93cd417a3493a80a2e026f773e105 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:18.435378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:18.446543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207961d-5f16-437c-84c1-442611482d05 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:18.526657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:18.528160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:18.555315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:18.556098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:18.556441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:18.556783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] PATH_INFO: `/v3/projects/8ffaef2488ee46a8b82c9452424a48a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:18.561235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffaef2488ee46a8b82c9452424a48a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:18.563277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:18.631003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffaef2488ee46a8b82c9452424a48a9', 'request_id': 'req-dc81271f-62be-46c3-ba43-225f703154b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.639864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f6f3-8d9b-4246-b5be-66679523b4e0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:18.646958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffaef2488ee46a8b82c9452424a48a9', 'request_id': 'req-dc81271f-62be-46c3-ba43-225f703154b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.656028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffaef2488ee46a8b82c9452424a48a9', 'request_id': 'req-dc81271f-62be-46c3-ba43-225f703154b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.675787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] The token cache is being invalidate because project 8ffaef2488ee46a8b82c9452424a48a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:18.676420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.677410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.679639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1119/2205] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:18 2022] DELETE /identity/v3/projects/8ffaef2488ee46a8b82c9452424a48a9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:18.684625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:18.687048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:18.689622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:18.740224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:18.752361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:18.752661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:18.752994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] PATH_INFO: `/v3/projects/518475493492462693b476f8dcd125d3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:18.754269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518475493492462693b476f8dcd125d3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:18.756352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:18.807493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81271f-62be-46c3-ba43-225f703154b2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:18.817009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:18.818759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:18.819190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:18.819481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] PATH_INFO: `/v3/users/b982c2860d4b42ccbbdf4f7c4aa932f7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:18.821175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b982c2860d4b42ccbbdf4f7c4aa932f7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:18.823918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:18.828201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:18.845177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518475493492462693b476f8dcd125d3', 'request_id': 'req-479ffd96-f499-46f6-bb81-1f99338464b1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.864675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518475493492462693b476f8dcd125d3', 'request_id': 'req-479ffd96-f499-46f6-bb81-1f99338464b1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.891444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518475493492462693b476f8dcd125d3', 'request_id': 'req-479ffd96-f499-46f6-bb81-1f99338464b1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.901320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b982c2860d4b42ccbbdf4f7c4aa932f7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.902483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b982c2860d4b42ccbbdf4f7c4aa932f7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.904427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] The token cache is being invalidate because project 518475493492462693b476f8dcd125d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:18.904816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.905564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.908351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1087/2206] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:18 2022] DELETE /identity/v3/projects/518475493492462693b476f8dcd125d3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:18.912514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:18.914360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:18.915380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b982c2860d4b42ccbbdf4f7c4aa932f7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.935455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b982c2860d4b42ccbbdf4f7c4aa932f7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.948890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b982c2860d4b42ccbbdf4f7c4aa932f7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:18.976435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1120/2207] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:18 2022] DELETE /identity/v3/users/b982c2860d4b42ccbbdf4f7c4aa932f7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:19.024190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ffd96-f499-46f6-bb81-1f99338464b1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:19.114928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:19.115994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:19.116576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:19.116930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] PATH_INFO: `/v3/users/a1ab3c5722b54e0d87aa8d78e481252a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:19.118431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ab3c5722b54e0d87aa8d78e481252a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:19.123402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:19.134489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:19.223376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ab3c5722b54e0d87aa8d78e481252a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.224499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ab3c5722b54e0d87aa8d78e481252a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.233359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ab3c5722b54e0d87aa8d78e481252a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.249618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ab3c5722b54e0d87aa8d78e481252a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.251412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ab3c5722b54e0d87aa8d78e481252a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.270005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1088/2208] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:18 2022] DELETE /identity/v3/users/a1ab3c5722b54e0d87aa8d78e481252a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:19.751839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:19.753873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:19.771146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bf3a4-765f-4d16-bb5c-f3974d64e572 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:19.866532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:19.867318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:19.867781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:19.868129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] PATH_INFO: `/v3/projects/29208945f4ca42e0a94fb2288f07de89` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:19.869430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29208945f4ca42e0a94fb2288f07de89)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:19.871987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:19.872197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:19.873999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:19.900062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18343f8a-80a8-441a-907b-92417c07f06f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:19.910517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:19.911181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:19.911464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:19.911775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] PATH_INFO: `/v3/projects/f2fbcd12a5744ae9bf6494bc7c673aee` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:19.913034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fbcd12a5744ae9bf6494bc7c673aee)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:19.914983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:19.968201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29208945f4ca42e0a94fb2288f07de89', 'request_id': 'req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.980702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29208945f4ca42e0a94fb2288f07de89', 'request_id': 'req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.990602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fbcd12a5744ae9bf6494bc7c673aee', 'request_id': 'req-2d3a0096-4685-4053-a0bd-97ed61185dfe'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:19.992433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29208945f4ca42e0a94fb2288f07de89', 'request_id': 'req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.004085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fbcd12a5744ae9bf6494bc7c673aee', 'request_id': 'req-2d3a0096-4685-4053-a0bd-97ed61185dfe'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.006643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] The token cache is being invalidate because project 29208945f4ca42e0a94fb2288f07de89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:20.007861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.008691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b7d34e1-cee4-48ba-80c3-d22d54d4ac04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.015094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1121/2209] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:19 2022] DELETE /identity/v3/projects/29208945f4ca42e0a94fb2288f07de89 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:20.015907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fbcd12a5744ae9bf6494bc7c673aee', 'request_id': 'req-2d3a0096-4685-4053-a0bd-97ed61185dfe'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.016653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a6c22-d0af-433a-9079-f96b90b6b6e2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.016945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a6c22-d0af-433a-9079-f96b90b6b6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.017233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a6c22-d0af-433a-9079-f96b90b6b6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.036548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] The token cache is being invalidate because project f2fbcd12a5744ae9bf6494bc7c673aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:20.037238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.037600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.039233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1089/2210] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:19 2022] DELETE /identity/v3/projects/f2fbcd12a5744ae9bf6494bc7c673aee => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:20.043248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.044949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.045855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-972a6c22-d0af-433a-9079-f96b90b6b6e2 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:20.047141 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-972a6c22-d0af-433a-9079-f96b90b6b6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.057019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:20.131704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1122/2211] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:20.139001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a0096-4685-4053-a0bd-97ed61185dfe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.139993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.142075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.142408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.144443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.145297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.145859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.146312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] PATH_INFO: `/v3/users/3902d8b666ce4414aea41538796d09f9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.148057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3902d8b666ce4414aea41538796d09f9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.150403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.160828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:20.191981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:20.194946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.216004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1123/2212] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:20.221217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.223653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3902d8b666ce4414aea41538796d09f9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.224338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3902d8b666ce4414aea41538796d09f9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.231496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2460b5d-7af5-4637-b162-d9b0e967940e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.231651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3902d8b666ce4414aea41538796d09f9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.241241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.241846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.242125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.242405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.243049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:20.243352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.243491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3902d8b666ce4414aea41538796d09f9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.244369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3902d8b666ce4414aea41538796d09f9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.245012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.258189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1090/2213] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:20 2022] DELETE /identity/v3/users/3902d8b666ce4414aea41538796d09f9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:20.261876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1124/2214] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:20.262662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.264136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.352467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15637-0e60-4ed8-9fff-065594fba023 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.361665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.362617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.362617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.362907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.363612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.365581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.406333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1091/2215] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:20.408589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.419579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c5054-78c8-48fc-abe2-fc820ab5a6d3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.502545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.503164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.503450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.503747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.504404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.506361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.541133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1125/2216] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:20.544833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.571974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145508-4f1a-4f77-a51e-7bc7a51b9a78 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.583231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.586579 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.603440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.604102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.604383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.604653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.605308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.606900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.613147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a35de-6bdb-482d-92d0-84b41722b864 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.619981 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.620890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.621362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.621825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] PATH_INFO: `/v3/projects/430981af748c4753acbda270141d8c28` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.623495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430981af748c4753acbda270141d8c28)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.625709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1092/2217] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:20.625890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.630092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.650223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f2b7f-0ea0-4977-8aa5-92275bc7f80b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.660346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.661378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.661678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.662022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.662954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.665500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.683020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1093/2218] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:20.687408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.690379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430981af748c4753acbda270141d8c28', 'request_id': 'req-59bf01d6-3664-49ef-b459-8e22d5a01739'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.698514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5838ef-94ff-4115-a78a-68fb146848c6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.701704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430981af748c4753acbda270141d8c28', 'request_id': 'req-59bf01d6-3664-49ef-b459-8e22d5a01739'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.711283 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430981af748c4753acbda270141d8c28', 'request_id': 'req-59bf01d6-3664-49ef-b459-8e22d5a01739'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.728386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] The token cache is being invalidate because project 430981af748c4753acbda270141d8c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:20.729030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.730045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-59bf01d6-3664-49ef-b459-8e22d5a01739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:20.732668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1126/2219] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:20 2022] DELETE /identity/v3/projects/430981af748c4753acbda270141d8c28 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:20.740180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf3c0-ef5f-4362-9e99-12343a33f72d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.740463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf3c0-ef5f-4362-9e99-12343a33f72d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.740739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cf3c0-ef5f-4362-9e99-12343a33f72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.753560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:20.755094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:20.774239 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-637cf3c0-ef5f-4362-9e99-12343a33f72d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:20.775740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.776323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-637cf3c0-ef5f-4362-9e99-12343a33f72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.776871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.777782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.778230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] PATH_INFO: `/v3/projects/28ef993fc04e4a7f955b4baedef155d1/users/dd2e7df162954564b353f1f45c6296f4/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.790772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1127/2220] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:20.800682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.800912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ef993fc04e4a7f955b4baedef155d1, user_id=dd2e7df162954564b353f1f45c6296f4, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.801045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.801269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.803852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432474-8c0a-4eb3-a610-92c3d72c0846 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.819445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1094/2221] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:20 2022] PUT /identity/v3/projects/28ef993fc04e4a7f955b4baedef155d1/users/dd2e7df162954564b353f1f45c6296f4/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:20.830517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.831333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.831689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.833663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:20.834907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.864007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] MFA Rules not processed for user `dd2e7df162954564b353f1f45c6296f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:20.866463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.931617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1128/2222] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:20.936121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.947154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6584f-eef1-418a-8603-ce7f3b12d119 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.953893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.954503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.955463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.956115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.956835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:20.957163 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:20.960216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:20.960354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1095/2223] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:20.971132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.972624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:20.976917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1129/2224] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:20.978181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:20.990844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4186b38-15b8-47e6-b79c-1b1a1776b086 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:20.996197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:20.997015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:20.997387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:20.997732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:20.998433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.000522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.031227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1130/2225] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:20 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:21.036350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.044458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef128b-b692-4383-89f3-ddd597d80a36 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.105324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.106256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.106735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.107186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.112043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f0c5e-729c-4335-acfa-077746bd6995 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.117679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.118323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.118856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.119132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.119874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.124603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.147095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:21.154541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1131/2226] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:21 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:21.158988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.173008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db00438-6e4b-41a9-aa7e-f85964763639 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.179578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.181089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.181512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.181782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.182433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.184056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.188337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.190192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.198373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1132/2227] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:21.202447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.207933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1096/2228] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:21.215151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd460c4-9b34-4145-8673-bf7e3105dc6a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.220129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.221522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.221853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.225601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.229532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.235641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.241507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.242470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.263731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1133/2229] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:21.265306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.265983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3940-c659-436f-9bfe-2daf79d3b69a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.276909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd9669-fe32-450d-bed9-08ec2457c672 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.277972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.278627 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.278910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.279185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] PATH_INFO: `/v3/users/dd2e7df162954564b353f1f45c6296f4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.282187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.282863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.283859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.284164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] PATH_INFO: `/v3/projects/41a8fb87cf0c4a9486a1a2d6ee6633e2/users/2975f18a381a4698abf35e2f624ad87d/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.288380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2e7df162954564b353f1f45c6296f4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.290322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.301211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a8fb87cf0c4a9486a1a2d6ee6633e2, user_id=2975f18a381a4698abf35e2f624ad87d, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.305763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164070a-e44c-4c08-9819-aab1f56fbaec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.335923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1134/2230] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:21 2022] PUT /identity/v3/projects/41a8fb87cf0c4a9486a1a2d6ee6633e2/users/2975f18a381a4698abf35e2f624ad87d/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:21.344031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.344906 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.346139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.369620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:21.390167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2e7df162954564b353f1f45c6296f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.390993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2e7df162954564b353f1f45c6296f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.399993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] MFA Rules not processed for user `2975f18a381a4698abf35e2f624ad87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:21.401246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.401669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2e7df162954564b353f1f45c6296f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.436450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2e7df162954564b353f1f45c6296f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.437875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b4fa14b9-03df-43b9-8975-ef4bc6868d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2e7df162954564b353f1f45c6296f4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.481487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1097/2231] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:21 2022] DELETE /identity/v3/users/dd2e7df162954564b353f1f45c6296f4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:21.487397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.487828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.488111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.512573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:21.514304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.518703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1135/2232] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:21.519749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.521412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.598921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674d31-e606-4e3b-bf4f-8bef7da6ef4c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.602287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1098/2233] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:21.622933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.626312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.680005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.680558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.680842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.681116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.691747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:21.711080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e21d0-1024-4896-aff9-afa3cc050439 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.716599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.717435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.717747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.718201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.731324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.733569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.753699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1099/2234] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:21.754638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.756684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5a9f9-f402-4e47-902f-4e3587a6dc38 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.767379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1136/2235] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:21.792541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:21.794821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:21.826582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf932e3-dee6-4333-a1e9-679443506554 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:21.832113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:21.833029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.833528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.833932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] PATH_INFO: `/v3/users/2975f18a381a4698abf35e2f624ad87d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.835383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2975f18a381a4698abf35e2f624ad87d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:21.837532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:21.898306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2975f18a381a4698abf35e2f624ad87d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.899016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2975f18a381a4698abf35e2f624ad87d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.908997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2975f18a381a4698abf35e2f624ad87d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.922798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2975f18a381a4698abf35e2f624ad87d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.924413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2975f18a381a4698abf35e2f624ad87d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:21.938490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1100/2236] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:21 2022] DELETE /identity/v3/users/2975f18a381a4698abf35e2f624ad87d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:21.943629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:21.943924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:21.944205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:21.969791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:21.977109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:21.978282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.042879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1137/2237] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:22.057539 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.060162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.085226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.087102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.103672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d381fa-0164-4f5a-97d5-598dfdc8e1d7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.112273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.113042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.113285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.113835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] PATH_INFO: `/v3/projects/28ef993fc04e4a7f955b4baedef155d1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.115214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ef993fc04e4a7f955b4baedef155d1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.117175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.127438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a907-9846-41ef-89fb-e9be3dc3d550 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.131212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.131864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.132144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.132465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.141848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.143332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.152909 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1101/2238] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:22.169884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ef993fc04e4a7f955b4baedef155d1', 'request_id': 'req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.179197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ef993fc04e4a7f955b4baedef155d1', 'request_id': 'req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.189267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ef993fc04e4a7f955b4baedef155d1', 'request_id': 'req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.201898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] The token cache is being invalidate because project 28ef993fc04e4a7f955b4baedef155d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:22.202190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.202873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b912d956-e210-43cc-bb3f-9e2e5f0ac7b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.219562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1138/2239] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:22 2022] DELETE /identity/v3/projects/28ef993fc04e4a7f955b4baedef155d1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:22.451042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.452477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.466494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ff630-7677-400d-a492-3a67491037b7 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.524822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.525460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.525736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.526005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] PATH_INFO: `/v3/projects/41a8fb87cf0c4a9486a1a2d6ee6633e2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.527125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a8fb87cf0c4a9486a1a2d6ee6633e2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.529091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.531491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:22.582701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a8fb87cf0c4a9486a1a2d6ee6633e2', 'request_id': 'req-6a425d89-22b5-41f2-b906-3b58d37c4d86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.589638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a8fb87cf0c4a9486a1a2d6ee6633e2', 'request_id': 'req-6a425d89-22b5-41f2-b906-3b58d37c4d86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.596643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a8fb87cf0c4a9486a1a2d6ee6633e2', 'request_id': 'req-6a425d89-22b5-41f2-b906-3b58d37c4d86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.607310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] The token cache is being invalidate because project 41a8fb87cf0c4a9486a1a2d6ee6633e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:22.607626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.608268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6a425d89-22b5-41f2-b906-3b58d37c4d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.609946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1102/2240] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:22 2022] DELETE /identity/v3/projects/41a8fb87cf0c4a9486a1a2d6ee6633e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:22.615367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.615560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.615827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.643551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:22.645127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.705785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1139/2241] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:22.712866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.713159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.713433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.733718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:22.734926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.743474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1103/2242] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:22.747738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.757810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04070b7-763b-4029-8fcc-2f9aaac7a6c8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.762641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.763281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.764124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.764402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.765079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:22.765383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.767003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.779817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1140/2243] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:22.785192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.794912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7dd36-ce60-4a09-882a-c074bfe16671 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.800123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.800724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.801003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.801268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.801898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.803801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.831899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1104/2244] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:22 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:22.836015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.846621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11681dae-f9c9-4d83-8560-02b23472e52b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.847125 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.848799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:22.859401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:22.907843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.908553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.908817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.909094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.909979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.911930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.915856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89771d6e-6583-4765-92af-7ff044620293 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.920996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.922008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.922308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.922580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] PATH_INFO: `/v3/users/8471e8f0923e4723ba148b941c83a377` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.924362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8471e8f0923e4723ba148b941c83a377)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.926614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.937161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1141/2245] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:22 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:22.938443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.948552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3919c-5bb6-4303-96c0-b83ab1ecd050 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.953361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.953944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.954221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.954515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.956130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.957517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.968972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1142/2246] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:22.972991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:22.974074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8471e8f0923e4723ba148b941c83a377'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.974733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8471e8f0923e4723ba148b941c83a377'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.981578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8471e8f0923e4723ba148b941c83a377'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.982729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7a465-89b5-406c-90a6-96d5772f4b01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:22.987367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:22.987980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:22.988256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:22.988529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:22.988947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:22.989151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:22.991300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:22.994893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8471e8f0923e4723ba148b941c83a377'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:22.995673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dc3bc56-bd79-49a1-91b2-24737b1450af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8471e8f0923e4723ba148b941c83a377'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.006043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1143/2247] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:23.007238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.009139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.010668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1105/2248] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:22 2022] DELETE /identity/v3/users/8471e8f0923e4723ba148b941c83a377 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:23.014742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.015029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.015300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.023040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727b79-e7fd-4827-b8dc-503d899ec01b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.034890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:23.036124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.084097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.084757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.085033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.085327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] PATH_INFO: `/v3/projects/37b09a81e18843779d164e4f7fc0fb2d/users/60a3b46d482f4cfcb53dcc55a865a2e6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.098348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1106/2249] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:23.100668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b09a81e18843779d164e4f7fc0fb2d, user_id=60a3b46d482f4cfcb53dcc55a865a2e6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.103295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521b1de-ffba-4bca-a37c-17b18ae86d12 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.108387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.110046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.117125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1144/2250] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:23 2022] PUT /identity/v3/projects/37b09a81e18843779d164e4f7fc0fb2d/users/60a3b46d482f4cfcb53dcc55a865a2e6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:23.128513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.128790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.129060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.156873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] MFA Rules not processed for user `60a3b46d482f4cfcb53dcc55a865a2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:23.158793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.171895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1582bc5-5b45-4925-b482-d4c707d5624b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.218348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:23.222180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.222791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.223079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.223361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.225881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1145/2251] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:23.230659 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.274274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:23.287660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.289397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.296359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819cb68-0a73-42f4-929c-c81e84ac5858 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.300057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1107/2252] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:23.301411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.302011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.302285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.302552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.303189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.305184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.338080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1146/2253] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:23.339608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.350622 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d665889-14ff-4c05-89f1-751af82f5257 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.410974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.411616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.411897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.412173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.412814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.414710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.436794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1108/2254] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:23.444661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.456502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa904001-7c1b-41be-b8f2-0bc600a08c89 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.461728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.462340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.462618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.462888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.463551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.465176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.476884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1147/2255] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:23.480732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.491498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7e648-1a37-4124-a106-f2b649147386 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.496492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.497095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.497376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.497645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.498313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.500055 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.513737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1109/2256] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:23.515092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.526307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af9ab7-950f-483d-a163-264b3e3bdfd3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.531251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.532088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.532680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.533624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] PATH_INFO: `/v3/projects/78af5acc8fc34b638e8eb5307cf03086/users/26facec0aedd4c73a0a08842e83f0719/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.550612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78af5acc8fc34b638e8eb5307cf03086, user_id=26facec0aedd4c73a0a08842e83f0719, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.553728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffbd0-512d-49bc-aa1a-d0c2eea61b46 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.568180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1148/2257] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:23 2022] PUT /identity/v3/projects/78af5acc8fc34b638e8eb5307cf03086/users/26facec0aedd4c73a0a08842e83f0719/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:23.578008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.579771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.579891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.580396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.580820 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.596794 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72933e-0462-4f21-95ba-9430ecfffcd3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.618521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] MFA Rules not processed for user `26facec0aedd4c73a0a08842e83f0719`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:23.619768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.667148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.667890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.668231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.668575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] PATH_INFO: `/v3/projects/e2a264c27225418d9169bb1f1707cb4a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.669768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a264c27225418d9169bb1f1707cb4a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.671764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.702439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1149/2258] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:23.707458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.719525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a16bc-3319-4b74-b925-009c74c7e698 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.726304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a264c27225418d9169bb1f1707cb4a', 'request_id': 'req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.735410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a264c27225418d9169bb1f1707cb4a', 'request_id': 'req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.743924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a264c27225418d9169bb1f1707cb4a', 'request_id': 'req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.750882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:23.758464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] The token cache is being invalidate because project e2a264c27225418d9169bb1f1707cb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:23.758906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.760302 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f3a223a1-0f40-4b6e-9e21-0b6a5f8dd98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:23.761948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1110/2259] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:23 2022] DELETE /identity/v3/projects/e2a264c27225418d9169bb1f1707cb4a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:23.766860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998127-780f-468d-97f2-cc514251b5a6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.767163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998127-780f-468d-97f2-cc514251b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.768005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998127-780f-468d-97f2-cc514251b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.768113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:23.789175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-25998127-780f-468d-97f2-cc514251b5a6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:23.790525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25998127-780f-468d-97f2-cc514251b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.796055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.796677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.796959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.797232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.797874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.799602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1111/2260] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:23.806009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.807859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.808448 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.808587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.833976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1150/2261] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:23.837692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.839603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.841590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:23.842805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:23.917072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1112/2262] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:23.917783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d680a-cb6e-4ff1-bb31-76d71db41b40 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.921287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.926474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.927997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.927997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.928640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.929927 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.932428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a154f-9937-479e-837c-384f0f9c526e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.932967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.937508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.938275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.938657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.939689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.940449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:23.940869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.942550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.963103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1113/2263] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:27:23.963360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1151/2264] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:23.965908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.967701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:23.979249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1487a9-a2b0-4e39-928d-833531d8a397 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.980067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29116828-937d-4424-9a0d-edd12c1f5fc0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:23.984506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.985289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.985667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.986763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.987775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.989476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:23.991462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:23.992562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:23.993066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:23.993560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:23.994748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:23.998508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.004319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1152/2265] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:24.005416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.015802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cf1b-0408-47d9-ac61-43c5397d5047 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.020949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.021782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.022068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.022356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] PATH_INFO: `/v3/projects/9d7569a8ad994211baa534a49cff77e1/users/51c475ddb01645c7a3d456f51b7d34b2/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.041701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1114/2266] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:23 2022] POST /identity/v3/projects => generated 357 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:24.041936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7569a8ad994211baa534a49cff77e1, user_id=51c475ddb01645c7a3d456f51b7d34b2, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.042851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.043316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.057428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1153/2267] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:24 2022] PUT /identity/v3/projects/9d7569a8ad994211baa534a49cff77e1/users/51c475ddb01645c7a3d456f51b7d34b2/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:24.065699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.114513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eec5f4-cc0d-4199-bfbb-231ec61b7cef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.136347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e915db-f9fa-4f96-89b3-0dd22fbe9e0d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.141562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.143105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.143573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.143856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.144505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.146099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.156081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1154/2268] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:24.161170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.170654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45806cf0-64e9-4442-a52c-d05fadd4c728 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.174787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.175518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.175678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.175962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.176178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.176287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.176455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.176724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] PATH_INFO: `/v3/projects/9d7569a8ad994211baa534a49cff77e1/users/51c475ddb01645c7a3d456f51b7d34b2/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.176847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.178648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7569a8ad994211baa534a49cff77e1, user_id=51c475ddb01645c7a3d456f51b7d34b2, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.178975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.182094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a610dfe-c108-4d7a-85bf-c6d54ab242b3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.188943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:24.189233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:24.195075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1155/2269] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:24 2022] PUT /identity/v3/projects/9d7569a8ad994211baa534a49cff77e1/users/51c475ddb01645c7a3d456f51b7d34b2/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:24.206911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1115/2270] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:24 2022] POST /identity/v3/users => generated 279 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:24.208344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.208635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.208908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.210643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.221498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4798c0-ad2a-46f8-b006-9fc8c0125de6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.238673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] MFA Rules not processed for user `51c475ddb01645c7a3d456f51b7d34b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:24.239827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.289200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.290863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.291150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.291424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.292593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.294216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.310470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1116/2271] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:24.311478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.314736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1156/2272] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:24.322387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c985ba5-8130-4c7c-a584-4bbb7299a9cc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.326617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.327307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.327942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.328071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.328226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.328498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.329184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.331382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.342692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1117/2273] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:24.347460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.357702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad291f1b-6f6e-433f-b5fb-4022cc9bbfdd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.363726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.364385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.364664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.364940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] PATH_INFO: `/v3/projects/393c418df7c04965b2cd897e8d27433e/users/3479322733884ddabbff0d35f68fca19/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.382796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=393c418df7c04965b2cd897e8d27433e, user_id=3479322733884ddabbff0d35f68fca19, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.385706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763a8d2-0f8a-4d8c-a0fe-b72d3ee65aab admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.389115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feda7ff-1abd-405b-96e2-66a5de732556 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.393619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.394402 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.394819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.395489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.398782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.403424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1118/2274] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:24 2022] PUT /identity/v3/projects/393c418df7c04965b2cd897e8d27433e/users/3479322733884ddabbff0d35f68fca19/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:24.415096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.415379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.415678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.447207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] MFA Rules not processed for user `3479322733884ddabbff0d35f68fca19`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:24.448446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.480312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.482096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.495567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1157/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:24.527354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1119/2276] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:27:24.563677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.565730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.576817 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657ccf-0af7-4532-a23b-4781f35265e7 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.624177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:24.647274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.647953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.648237 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.648515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.658988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.660855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e758f2-7433-4c71-8919-3eebd68cf530 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.671977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1158/2277] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:24.690341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:24.756667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:24.778614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee19ab-e660-40b1-a690-c653a6119ea9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:24.784832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:24.785624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.795702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.795702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] PATH_INFO: `/v3/users/3479322733884ddabbff0d35f68fca19` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.795702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3479322733884ddabbff0d35f68fca19)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:24.795702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:24.867701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3479322733884ddabbff0d35f68fca19'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:24.867701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3479322733884ddabbff0d35f68fca19'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:24.867701 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3479322733884ddabbff0d35f68fca19'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:24.882326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3479322733884ddabbff0d35f68fca19'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:24.883058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3479322733884ddabbff0d35f68fca19'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:24.897470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1120/2278] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:24 2022] DELETE /identity/v3/users/3479322733884ddabbff0d35f68fca19 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:24.902997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:24.903521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:24.903804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:24.927392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:24.928856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:24.994000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1159/2279] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:25.005332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.006844 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:25.067990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441026-5491-46ca-a899-1d6d22796d73 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.071711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.072317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.072598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.072874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.082399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.083864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91d319-85f2-465b-9b5a-e285863ba254 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.093883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1121/2280] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:25.451501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.453046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:25.471637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d74855-d6dc-4231-ab15-a66b73e27cf6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.477014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.477375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.477658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.477932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] PATH_INFO: `/v3/projects/393c418df7c04965b2cd897e8d27433e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.479090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=393c418df7c04965b2cd897e8d27433e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.481129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.530709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393c418df7c04965b2cd897e8d27433e', 'request_id': 'req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:25.538081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393c418df7c04965b2cd897e8d27433e', 'request_id': 'req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:25.544039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393c418df7c04965b2cd897e8d27433e', 'request_id': 'req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:25.555848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] The token cache is being invalidate because project 393c418df7c04965b2cd897e8d27433e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:25.556140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:25.556791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-eb5cf0b8-9039-424d-aeae-c8ed2d320f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:25.558324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1160/2281] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:25 2022] DELETE /identity/v3/projects/393c418df7c04965b2cd897e8d27433e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:25.563226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.563713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.563881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.584350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:25.585540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:25.646686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1122/2282] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:25.654400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4650-0601-493f-b401-748119e5712f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.654681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4650-0601-493f-b401-748119e5712f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.654952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4650-0601-493f-b401-748119e5712f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.676008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-70fd4650-0601-493f-b401-748119e5712f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:25.677342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-70fd4650-0601-493f-b401-748119e5712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:25.685704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1161/2283] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:25.690326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.700097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30523c1-0840-49fa-b3cd-39dafc981996 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.705130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.706962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.707320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.707707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.709014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:25.709320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.711349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.713001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:25.728334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1123/2284] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:27:25.732910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd4650-0601-493f-b401-748119e5712f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.742317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd4650-0601-493f-b401-748119e5712f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.746923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.747514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.747816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.748085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.748721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.750609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.777319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1162/2285] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:25 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:25.781513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.792331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f367c3-4c6d-4a90-9c22-1dc8adbda520 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.848218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.848849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.849129 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.849402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.850045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.852056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.874811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1124/2286] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:25 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:25.879896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.897778 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9bca7-862a-45dd-9d92-ae8095c1a8c3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.902595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.903223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.903795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.903895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.904464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.906066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.917879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1163/2287] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:25.922096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.935272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2649f-edb1-4cb7-a209-05a26aa8ddcd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.940519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.941121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.941454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.941727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:25.942375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:25.943985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadcaf3-700c-473e-844d-5c197893e0e8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:25.958034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1125/2288] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:25.962276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:25.973714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31f893-284b-4b25-9483-1a6eb77911f0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:25.991308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:25.992021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:25.992301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:25.992599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] PATH_INFO: `/v3/projects/62283ebae9464c0bb0047a2b196b361f/users/2b135603776d4d26a8696500a039b999/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:26.009876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=62283ebae9464c0bb0047a2b196b361f, user_id=2b135603776d4d26a8696500a039b999, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:26.012720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:26.028642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1164/2289] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:25 2022] PUT /identity/v3/projects/62283ebae9464c0bb0047a2b196b361f/users/2b135603776d4d26a8696500a039b999/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:26.039697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:26.039972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:26.040186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:26.101798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] MFA Rules not processed for user `2b135603776d4d26a8696500a039b999`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:26.102784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:26.146769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:26.148481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:26.153784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:26.191576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1126/2290] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:26.202388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:26.203969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:26.216173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f7e8-efc6-4366-b931-589a24b5cabf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:26.218721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a381-d7ba-4f5b-982b-cb08a939e3ef None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:26.220818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:26.221692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:26.221974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:26.222254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:26.223788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:26.224046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:26.224395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:26.224672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:26.224943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:26.235689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:26.238159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:26.248507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1127/2291] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:26.296533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:26.298278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:26.309050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1165/2292] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:26.876360 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:26.885399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cbd5-8c96-4ff5-b9a2-449c50123504 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:26.888971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:26.889553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:26.889826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:26.890092 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:26.891811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:26.940916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:26.960280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:26.961708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270c7cc-f97e-4adf-8514-9fe870f46119 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:26.971762 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1128/2293] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:28.881391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:28.943056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6ca5-ccfa-4aca-b09c-6fd8d09f350a service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:28.947633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:28.948256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:28.948537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:28.948811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] PATH_INFO: `/v3/users/2b135603776d4d26a8696500a039b999` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:28.949931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b135603776d4d26a8696500a039b999)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:28.951888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:28.996997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b135603776d4d26a8696500a039b999'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:28.997642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b135603776d4d26a8696500a039b999'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.004707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b135603776d4d26a8696500a039b999'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.006555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:29.018176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b135603776d4d26a8696500a039b999'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.018889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b135603776d4d26a8696500a039b999'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.031245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1166/2294] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:28 2022] DELETE /identity/v3/users/2b135603776d4d26a8696500a039b999 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:29.036697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.037231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.037359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.056996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:29.058208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:29.116470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1129/2295] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:29.126535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:29.128290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:29.186857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3349-e7bf-4bf3-b58e-a4a245b71072 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:29.190925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:29.191527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.191834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.192108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.201069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:29.202516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cee65f-4537-40bf-8240-8c59c953cd02 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:29.212408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1167/2296] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:29.495039 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:29.496588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:29.509641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b19f-2f93-4457-ba26-814d18dfe28f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:29.520789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:29.521660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.521943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.522212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] PATH_INFO: `/v3/projects/62283ebae9464c0bb0047a2b196b361f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.523615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62283ebae9464c0bb0047a2b196b361f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:29.525773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:29.583260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62283ebae9464c0bb0047a2b196b361f', 'request_id': 'req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.592294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62283ebae9464c0bb0047a2b196b361f', 'request_id': 'req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.602296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62283ebae9464c0bb0047a2b196b361f', 'request_id': 'req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.614287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] The token cache is being invalidate because project 62283ebae9464c0bb0047a2b196b361f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:29.614710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.615406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e3f13fe-ac6c-47c8-a707-2aeef55500e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:29.617119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1130/2297] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:29 2022] DELETE /identity/v3/projects/62283ebae9464c0bb0047a2b196b361f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:29.625713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.626004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.626287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.659119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:29.661143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:29.748828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1168/2298] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:29.754296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.754635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.754916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.785424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:29.786654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:29.802606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1131/2299] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:29.806863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:29.822744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745ab2-aec9-49f5-a928-fe731985ca00 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:29.829184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:29.830429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.830803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.831203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.831983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:29.832407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:29.834075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:29.846713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1169/2300] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:29.851314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:29.860737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9052a9-815a-4a27-852b-606328864e97 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:29.866708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:29.867716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.867998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.868266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.868904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:29.870954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:29.907681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1132/2301] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:29 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:29.908858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:29.919364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bf99b-9802-4c6b-8f5b-3d7caf81a155 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:29.981739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:29.982353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:29.982643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:29.982921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:29.983588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:29.986257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:29.998683 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:30.015050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1170/2302] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:29 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:30.019498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:30.030250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8ff2-c065-4ba5-92bf-bb5522d2e271 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:30.036114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:30.037105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:30.037642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:30.037741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:30.038722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:30.041392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:30.056263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1133/2303] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:30.060492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:30.071990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39756f69-0908-47f9-af2b-243f484b7f0b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:30.076679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:30.077281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:30.077564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:30.077841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:30.078487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:30.080164 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b0eca-5716-44a6-af96-4ac93c8a651d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:30.090746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1171/2304] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:30.095134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:30.105728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5dc66-bb00-447b-bcdf-7fee5c407085 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:30.110516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:30.111758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:30.112040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:30.112330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] PATH_INFO: `/v3/projects/94fb247cca724d63bf21a4b138899c27/users/3e8d25b3cccd4e5999dc044faf1a9bfc/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:30.129059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fb247cca724d63bf21a4b138899c27, user_id=3e8d25b3cccd4e5999dc044faf1a9bfc, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:30.133931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:30.150618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1134/2305] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:30 2022] PUT /identity/v3/projects/94fb247cca724d63bf21a4b138899c27/users/3e8d25b3cccd4e5999dc044faf1a9bfc/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:30.161191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:30.161486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:30.161771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:30.191786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] MFA Rules not processed for user `3e8d25b3cccd4e5999dc044faf1a9bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:30.193007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:30.263678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1172/2306] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:30.274561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:30.276353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:30.281758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:30.338836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128082cd-d8de-4a1d-ba06-d7b5ab48b60b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:30.342674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:30.343296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:30.343614 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:30.343893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:30.353328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:30.354753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19cc9e-876f-4c7e-8429-7da62ce17ef8 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:30.365126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1135/2307] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:30.927774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:30.929226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:30.997998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bfb0c-1ce2-4e2a-91da-bb0e83e5e119 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:31.002526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:31.003144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:31.003420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:31.003721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] PATH_INFO: `/v3/users/60a3b46d482f4cfcb53dcc55a865a2e6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:31.004845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a3b46d482f4cfcb53dcc55a865a2e6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:31.006744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:31.051926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a3b46d482f4cfcb53dcc55a865a2e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.052566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a3b46d482f4cfcb53dcc55a865a2e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.059663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a3b46d482f4cfcb53dcc55a865a2e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.073027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a3b46d482f4cfcb53dcc55a865a2e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.073680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a3b46d482f4cfcb53dcc55a865a2e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.076223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:31.088744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1173/2308] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:30 2022] DELETE /identity/v3/users/60a3b46d482f4cfcb53dcc55a865a2e6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:31.093166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:31.093620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:31.093711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:31.112469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:31.113664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:31.172174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1136/2309] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:31.181845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:31.183223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:31.239280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7952d-d6a8-4257-939c-9e2c04a554ba admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:31.242793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:31.243388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:31.243700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:31.243969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:31.253306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:31.254720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:31.263957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1174/2310] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:31.536758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:31.538181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:31.558063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c26f7-98bf-43cb-a363-865799c360a5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:31.563076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:31.563772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:31.564852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:31.565260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] PATH_INFO: `/v3/projects/37b09a81e18843779d164e4f7fc0fb2d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:31.566254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b09a81e18843779d164e4f7fc0fb2d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:31.568199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:31.613913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b09a81e18843779d164e4f7fc0fb2d', 'request_id': 'req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.620135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b09a81e18843779d164e4f7fc0fb2d', 'request_id': 'req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.627401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b09a81e18843779d164e4f7fc0fb2d', 'request_id': 'req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.638329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] The token cache is being invalidate because project 37b09a81e18843779d164e4f7fc0fb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:31.638444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.639036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.640607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1137/2311] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:31 2022] DELETE /identity/v3/projects/37b09a81e18843779d164e4f7fc0fb2d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:31.644961 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:31.654697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e8c33-8faa-4ea8-828f-9d166f546051 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:31.714426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:31.715105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:31.715381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:31.717135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] PATH_INFO: `/v3/users/26facec0aedd4c73a0a08842e83f0719` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:31.718150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=26facec0aedd4c73a0a08842e83f0719)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:31.720171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:31.770019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26facec0aedd4c73a0a08842e83f0719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.771064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26facec0aedd4c73a0a08842e83f0719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.778040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26facec0aedd4c73a0a08842e83f0719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.790308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26facec0aedd4c73a0a08842e83f0719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.790972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26facec0aedd4c73a0a08842e83f0719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:31.806529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1175/2312] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:31 2022] DELETE /identity/v3/users/26facec0aedd4c73a0a08842e83f0719 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:32.193474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:32.195313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:32.210008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ba1cd-8837-4669-bf8b-7fa83a93ce53 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:32.289174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:32.325253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:32.325889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:32.326266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:32.326439 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] PATH_INFO: `/v3/projects/78af5acc8fc34b638e8eb5307cf03086` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:32.327631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78af5acc8fc34b638e8eb5307cf03086)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:32.329556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:32.377476 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78af5acc8fc34b638e8eb5307cf03086', 'request_id': 'req-3d023d65-b134-4c25-8ce5-c5e68c285ce5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.384318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78af5acc8fc34b638e8eb5307cf03086', 'request_id': 'req-3d023d65-b134-4c25-8ce5-c5e68c285ce5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.390616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78af5acc8fc34b638e8eb5307cf03086', 'request_id': 'req-3d023d65-b134-4c25-8ce5-c5e68c285ce5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.401379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] The token cache is being invalidate because project 78af5acc8fc34b638e8eb5307cf03086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:32.401693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.402312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.403843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1138/2313] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:32 2022] DELETE /identity/v3/projects/78af5acc8fc34b638e8eb5307cf03086 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:32.408430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:32.410018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:32.424887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8a0-4570-41f9-b9c0-70e23b66a6fa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:32.437024 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:32.484098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:32.484721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:32.484998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:32.485270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] PATH_INFO: `/v3/users/51c475ddb01645c7a3d456f51b7d34b2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:32.486890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c475ddb01645c7a3d456f51b7d34b2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:32.490459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:32.539634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c475ddb01645c7a3d456f51b7d34b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.540190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c475ddb01645c7a3d456f51b7d34b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.547068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c475ddb01645c7a3d456f51b7d34b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.559833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c475ddb01645c7a3d456f51b7d34b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.560478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-60adf396-d12e-4ff6-8f2e-7140a9638952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c475ddb01645c7a3d456f51b7d34b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.573871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1176/2314] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:32 2022] DELETE /identity/v3/users/51c475ddb01645c7a3d456f51b7d34b2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:32.852151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:32.853704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:32.868771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d023d65-b134-4c25-8ce5-c5e68c285ce5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:32.930135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:32.930776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:32.931043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:32.931310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] PATH_INFO: `/v3/projects/9d7569a8ad994211baa534a49cff77e1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:32.932516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7569a8ad994211baa534a49cff77e1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:32.934515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:32.983338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7569a8ad994211baa534a49cff77e1', 'request_id': 'req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.991785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7569a8ad994211baa534a49cff77e1', 'request_id': 'req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.998453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7569a8ad994211baa534a49cff77e1', 'request_id': 'req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:32.999818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:33.013912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] The token cache is being invalidate because project 9d7569a8ad994211baa534a49cff77e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:33.014205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:33.014870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b0f9dbc0-5d6f-4bee-a184-398d0e39de27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:33.016372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1139/2315] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:32 2022] DELETE /identity/v3/projects/9d7569a8ad994211baa534a49cff77e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:33.021440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.021744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.022114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.042472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:33.044234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:33.136573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1177/2316] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:33.145029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.145200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.145489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.180979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:33.182298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:33.191651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1140/2317] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:33.195839 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.206583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e958f2-2676-4af8-8903-753b2efbc383 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.211564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.212186 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.212466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.212744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.213503 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:33.213809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.215430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.229993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1178/2318] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:33.234329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.245170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6067177-6f7b-4f0b-96b9-4a27077b4e05 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.250097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.250708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.250986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.251320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.252139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.254950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.289623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1141/2319] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:33 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:33.293900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.309611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b762d8-250b-4fdf-bd82-9b52a53bbb10 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.337868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:33.389575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.390187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.390464 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.391151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.397514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.399597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.444502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1179/2320] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:33 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:33.445826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.459721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afb459-32ba-4434-82d5-47ccbc9fe51b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.464507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.465112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.465390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.465648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.466298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.468158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.496511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1142/2321] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:33.500854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.512044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd40a9-706d-4450-a2bf-77f3189f9303 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.518610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.519298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.519776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.520146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.520859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.522529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d967a-6598-42c8-864a-279b2ae417fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.535702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1180/2322] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:33.540564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.555252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b217a-e898-4588-b58f-1f56ce15131b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.559979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.560816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.561101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.561402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] PATH_INFO: `/v3/projects/0b8a88d8ba6c49bdaa32f75ab173341f/users/75f1f002d7064ec29cbde5bef1307821/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.576879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8a88d8ba6c49bdaa32f75ab173341f, user_id=75f1f002d7064ec29cbde5bef1307821, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.581059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.594236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1143/2323] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:33 2022] PUT /identity/v3/projects/0b8a88d8ba6c49bdaa32f75ab173341f/users/75f1f002d7064ec29cbde5bef1307821/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:33.608556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1b6c-14d4-4528-91c3-6f56c0ab6906 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.608846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1b6c-14d4-4528-91c3-6f56c0ab6906 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.609153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1b6c-14d4-4528-91c3-6f56c0ab6906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.637912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b15d1b6c-14d4-4528-91c3-6f56c0ab6906 None None] MFA Rules not processed for user `75f1f002d7064ec29cbde5bef1307821`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:33.639107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b15d1b6c-14d4-4528-91c3-6f56c0ab6906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:33.702765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1181/2324] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:33.714903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:33.716686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:33.779454 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af922-5500-4468-8f25-f77d3d8ebd89 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:33.783118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:33.783744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:33.784028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:33.784565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:33.792956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:33.794382 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:33.805524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1144/2325] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:33 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:34.311193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffcf4-734f-4258-9d77-93d2f826d755 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:34.311484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffcf4-734f-4258-9d77-93d2f826d755 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:34.311796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffcf4-734f-4258-9d77-93d2f826d755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:34.331600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8aeffcf4-734f-4258-9d77-93d2f826d755 None None] MFA Rules not processed for user `ba45eee6687b45158555ef5f0c595680`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:34.332722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8aeffcf4-734f-4258-9d77-93d2f826d755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:34.391175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1182/2326] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:43.801013 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:44.384641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:47.199396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:47.268032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b481-715a-4fab-9846-bf6f17659140 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:47.272728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:47.273361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:47.273668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:47.274067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] PATH_INFO: `/v3/users/75f1f002d7064ec29cbde5bef1307821` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:47.275221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f1f002d7064ec29cbde5bef1307821)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:47.277410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:47.325317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f1f002d7064ec29cbde5bef1307821'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.325963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f1f002d7064ec29cbde5bef1307821'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.333373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f1f002d7064ec29cbde5bef1307821'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.348893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f1f002d7064ec29cbde5bef1307821'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.349584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f1f002d7064ec29cbde5bef1307821'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.362802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1145/2327] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:27:47 2022] DELETE /identity/v3/users/75f1f002d7064ec29cbde5bef1307821 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:47.367373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:47.367940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:47.368038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:47.387772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:47.388971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:47.453840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1183/2328] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:47.464661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:47.466194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:47.536912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e586aa-dead-407f-8413-3507b9e550fe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:47.540560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:47.541217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:47.541498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:47.541767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:47.552327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:47.554273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fc4a0-38ff-4b82-b138-e38d3dab4a8f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:47.564634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1146/2329] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:27:47.839114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:47.840826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:47.854859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc8a65-1362-487a-98fc-16121f9e9374 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:47.859587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:47.860218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:47.861132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:47.861405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] PATH_INFO: `/v3/projects/0b8a88d8ba6c49bdaa32f75ab173341f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:47.862710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8a88d8ba6c49bdaa32f75ab173341f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:47.864655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:47.912076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8a88d8ba6c49bdaa32f75ab173341f', 'request_id': 'req-553dc5e4-0732-4354-ba34-be14dc018591'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.920015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8a88d8ba6c49bdaa32f75ab173341f', 'request_id': 'req-553dc5e4-0732-4354-ba34-be14dc018591'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.926435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8a88d8ba6c49bdaa32f75ab173341f', 'request_id': 'req-553dc5e4-0732-4354-ba34-be14dc018591'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.937438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] The token cache is being invalidate because project 0b8a88d8ba6c49bdaa32f75ab173341f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:27:47.937729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.939090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-553dc5e4-0732-4354-ba34-be14dc018591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:27:47.940641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1184/2330] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:27:47 2022] DELETE /identity/v3/projects/0b8a88d8ba6c49bdaa32f75ab173341f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:47.945941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:47.946310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:47.946653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:47.966244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:47.967463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:47.984935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:48.030399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1147/2331] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:48.038187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.039349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.039732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.060532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:48.061747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:48.070355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1185/2332] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:48.074608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.084261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0ea2f-5eed-4aac-bb91-caef960e259c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.088925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.089522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.089800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.090075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.090725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:27:48.091148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.092781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.104408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1148/2333] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:27:48.108700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.118566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f923e-aeb8-46b2-ada4-bc2adc2dfe10 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.123172 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.123802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.124080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.124350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.124983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.127525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.154640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1186/2334] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:48 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:48.160083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.170481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e42719-44ea-4b1f-b1b6-c9dbecec6178 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.230637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.231290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.231585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.231863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.232496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.234652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.258107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1149/2335] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:48 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:48.262401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.273413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bef95-3d2d-4126-b3e2-91bcdda3bb92 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.277984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.278571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.278846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.279136 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.279780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.281425 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.292336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1187/2336] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:48.296259 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.306204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582c98a-173d-4271-b792-2bc3b7ed122d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.310896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.311489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.311809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.312084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.312726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.315074 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911200-9835-48c5-8706-54f85bfa8265 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.325544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1150/2337] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:48.330338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.340598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b142b3d-ba40-47f3-b754-bd70dc3c93a4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.345995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.346656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.346945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.347223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] PATH_INFO: `/v3/projects/e46e2167ee15481fab472e5b101c9c60/users/469561e39c274301b16677d12cac3a67/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.362977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46e2167ee15481fab472e5b101c9c60, user_id=469561e39c274301b16677d12cac3a67, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.365629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.379522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1188/2338] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:48 2022] PUT /identity/v3/projects/e46e2167ee15481fab472e5b101c9c60/users/469561e39c274301b16677d12cac3a67/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:27:48.384375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0f91-a5a9-4524-960d-12de28576ff8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.384669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0f91-a5a9-4524-960d-12de28576ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.384923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0f91-a5a9-4524-960d-12de28576ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.404380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-19fa0f91-a5a9-4524-960d-12de28576ff8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:48.405645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-19fa0f91-a5a9-4524-960d-12de28576ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:48.464167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1151/2339] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:48.475063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:48.476880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:48.498816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:48.537387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc3392-7c21-4403-a83e-ddbcae4e4ebe admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:48.541369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:48.542102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:48.542386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:48.542665 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:48.552746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:48.554192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:48.564420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1189/2340] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:27:54.079782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.080488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.080488 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.130076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] MFA Rules not processed for user `469561e39c274301b16677d12cac3a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:27:54.134105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:27:54.184323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:27:54.318717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1152/2341] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:27:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:27:54.322841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.333851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bd4bb-f95c-481a-8619-796ed0090048 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.339608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.340226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.340507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.340779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.341466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.351339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.396990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1190/2342] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:27:54 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:27:54.400793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.414431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e982f3-170c-4275-856a-9f54db62e88b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.485960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.486577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.486862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.487136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.487836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.489779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.512634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1153/2343] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:27:54 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:27:54.517161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.527984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a6855-64e9-4970-a851-a6a3528b7d4c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.532960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.533577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.533857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.534390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.535033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.536663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.548761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1191/2344] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:27:54.553036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.564783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be45d8-50bf-4f8b-a18f-72e8b018a155 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.569949 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.570569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.570845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.571119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] PATH_INFO: `/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4/users/d6ffc0b8319b47b8b96fb6d591097963/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.593997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d750c69f7c364c5ab2f5d2541b8430e4, user_id=d6ffc0b8319b47b8b96fb6d591097963, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.597344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.618720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1154/2345] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:54 2022] PUT /identity/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4/users/d6ffc0b8319b47b8b96fb6d591097963/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:27:54.622542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.640326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf85a8f-260d-461f-ab09-36c9e0872793 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.727340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.728105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.728343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.728670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.729340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.730966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3fe55-16cb-484f-9c45-ec125cb52180 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.744957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1192/2346] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:27:54.749416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:27:54.763957 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6a10-ac77-4498-8dc7-7a4af06b9f59 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:27:54.769281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:27:54.769959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:27:54.770233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:27:54.770503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] PATH_INFO: `/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4/users/d6ffc0b8319b47b8b96fb6d591097963/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:27:54.772767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d750c69f7c364c5ab2f5d2541b8430e4, user_id=d6ffc0b8319b47b8b96fb6d591097963, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:27:54.775430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:27:54.788777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1155/2347] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:27:54 2022] PUT /identity/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4/users/d6ffc0b8319b47b8b96fb6d591097963/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:28:01.101775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5802f8-b0dc-4ed7-98e8-683ec60c6dbe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:01.102308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5802f8-b0dc-4ed7-98e8-683ec60c6dbe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:01.102308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5802f8-b0dc-4ed7-98e8-683ec60c6dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:01.161931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca5802f8-b0dc-4ed7-98e8-683ec60c6dbe None None] MFA Rules not processed for user `d6ffc0b8319b47b8b96fb6d591097963`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:01.163313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca5802f8-b0dc-4ed7-98e8-683ec60c6dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:01.254727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1193/2348] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:01.269799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:01.272166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:01.365164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4861e4-c32a-44c1-9c02-e827968505dc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:01.373403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:01.374059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:01.374334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:01.374601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:01.381306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:01.454966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:01.456717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037ca4e-2c3c-4be5-9b1f-63aad935ed6d service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:01.466408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1156/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:28:02.769662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:02.770169 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:02.770277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:02.791210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] MFA Rules not processed for user `469561e39c274301b16677d12cac3a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:02.792632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:02.800542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1194/2350] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:11.462081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=24 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:12.785137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:14.477926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:14.478920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:14.479308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:14.500195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:14.501445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:14.573060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1157/2351] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:14.617234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:14.618635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:14.692786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c00f9-cde8-43c2-9443-408c740423cf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:14.697916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:14.698519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:14.698796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:14.699064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:14.708386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:14.709992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b335d-b184-4e91-a83a-058bca32b69c service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:14.718483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1195/2352] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:28:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:28:23.033661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:23.035621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:23.192327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06b75f-4224-4f0f-95c7-d2f12c6b76ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:23.199277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:23.200241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:23.200643 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:23.200944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:23.203253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:23.220756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:23.223011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471ffa-c684-458e-a674-20dbeee9ae50 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:23.238973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1158/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:28:24.715177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=32 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:29.019647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:29.020677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:29.021116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:29.042023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] MFA Rules not processed for user `0d773c7f6aca4ea2ad2872f67e9116af`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:29.043389 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:29.138514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1196/2354] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:33.232224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=57 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:37.674264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:37.674904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:37.674904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:37.694944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:37.696160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:37.703308 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1159/2355] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:37.717082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:37.729543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1193-4c3c-460b-a00e-286426e5a2a2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:37.734680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:37.735221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:37.736242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:37.736679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:37.744604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:37.746196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:37.755057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1197/2356] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:28:40.130154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:40.131876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:40.261261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26919d7-f963-4bc5-9a48-3148ea71e922 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:40.272967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:40.274027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:40.274474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:40.274914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] PATH_INFO: `/v3/users/ba45eee6687b45158555ef5f0c595680` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:40.276291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba45eee6687b45158555ef5f0c595680)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:40.278835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:40.339473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba45eee6687b45158555ef5f0c595680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.340221 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba45eee6687b45158555ef5f0c595680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.351380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba45eee6687b45158555ef5f0c595680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.377977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba45eee6687b45158555ef5f0c595680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.379869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e4649a9-470f-4d73-bf45-9bf485719e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba45eee6687b45158555ef5f0c595680'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.420871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1160/2357] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:28:40 2022] DELETE /identity/v3/users/ba45eee6687b45158555ef5f0c595680 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:28:40.617489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:40.639767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ed9a3-849a-45cb-8fe3-a4e4f750f512 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:40.710418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:40.711162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:40.711634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:40.711917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] PATH_INFO: `/v3/projects/bdf0d6145a4848a8bc624ded77bc27ac` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:40.713181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf0d6145a4848a8bc624ded77bc27ac)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:40.721984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:40.782078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf0d6145a4848a8bc624ded77bc27ac', 'request_id': 'req-c1120d82-39ea-4525-95ed-d914cbdea6f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.798774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf0d6145a4848a8bc624ded77bc27ac', 'request_id': 'req-c1120d82-39ea-4525-95ed-d914cbdea6f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.802401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf0d6145a4848a8bc624ded77bc27ac', 'request_id': 'req-c1120d82-39ea-4525-95ed-d914cbdea6f9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.815053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] The token cache is being invalidate because project bdf0d6145a4848a8bc624ded77bc27ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:28:40.815418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.816193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c1120d82-39ea-4525-95ed-d914cbdea6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:40.817787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1198/2358] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:28:40 2022] DELETE /identity/v3/projects/bdf0d6145a4848a8bc624ded77bc27ac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:28:40.823472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:40.824168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:40.824553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:40.846696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:40.846973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:40.916868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1161/2359] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:40.924591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:40.924854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:40.925130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:40.955429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:40.957170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:40.964847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1199/2360] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:40.969045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:40.979348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd933a-c84a-4424-9070-642e8e7654ca None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:40.984106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:40.984714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:40.985783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:40.985971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:40.986580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:28:40.986882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:40.988498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:40.995308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:40.996835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.001977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:41.006938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1162/2361] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:28:41.011322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.011928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190901a-93ad-4479-9597-0d0a3c904791 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.016723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.017960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.018416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.018846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] PATH_INFO: `/v3/users/3e8d25b3cccd4e5999dc044faf1a9bfc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.021335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8d25b3cccd4e5999dc044faf1a9bfc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.023504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.025566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545526a-1f64-4315-829b-58850b1fcebc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.040194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:41.041214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.041876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.042159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.042644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.043636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.045543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.088968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1163/2362] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:28:41 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:28:41.093235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.122497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8d25b3cccd4e5999dc044faf1a9bfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.123440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8d25b3cccd4e5999dc044faf1a9bfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.137035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8d25b3cccd4e5999dc044faf1a9bfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.151575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8d25b3cccd4e5999dc044faf1a9bfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.152491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b74d4ea6-1f88-4941-8e71-bb717a7f11c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8d25b3cccd4e5999dc044faf1a9bfc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.168828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1200/2363] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:28:40 2022] DELETE /identity/v3/users/3e8d25b3cccd4e5999dc044faf1a9bfc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:28:41.172908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.173753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.174028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.180553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f40a-bf32-43b1-beb2-78fe120d11ef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.194285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:41.195494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.240763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.241374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.241657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.241934 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.242556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.244504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.256227 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1201/2364] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:28:41.266015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.268142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.271128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1164/2365] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:28:41 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:28:41.275255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.276895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.289809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9a469-ef8a-4d2e-98c5-a2a6552149d2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.294559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.295374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.295689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.295965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.296616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.298764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.308914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1165/2366] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:28:41.312714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.323254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646b42-663e-43fc-aaf5-46f6176c4543 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.328709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.329894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.330177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.330450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.331799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.331918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d0fd3-1072-496e-993f-58251be2cb54 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.332912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.336420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.337057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.337492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.337771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.343947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1166/2367] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:28:41.348555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.349182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.350633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4398a00-b640-4eca-bdc0-2832aa588be4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.358788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19c8e3-13a1-4b84-90d0-632159e12c6c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.360628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1202/2368] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:28:41.372633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.373405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.373755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.374108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] PATH_INFO: `/v3/projects/767cd74e36bf4ebb8e30b6f82d2a9164/users/95558d9a28bf4a0bac9dd72445999781/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.394545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] RBAC: Authorizing `identity:create_grant(project_id=767cd74e36bf4ebb8e30b6f82d2a9164, user_id=95558d9a28bf4a0bac9dd72445999781, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.397368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.440444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1167/2369] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:28:41 2022] PUT /identity/v3/projects/767cd74e36bf4ebb8e30b6f82d2a9164/users/95558d9a28bf4a0bac9dd72445999781/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:28:41.451185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.451591 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.451938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.513980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] MFA Rules not processed for user `95558d9a28bf4a0bac9dd72445999781`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:41.516657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.543324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:41.622853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1203/2370] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:41.637897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.639994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.687059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:41.715841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:41.717352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:41.757268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01af0d0-978e-4015-8632-3eda881d113f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.765215 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.765931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.766265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.766469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.778791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.780985 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780ecc0-1233-4146-a993-04f6d6bf8ac1 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.793107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1168/2371] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:28:41.828638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca269-36c8-4e84-9fbf-a6e265b123a3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:41.834841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:41.835714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.836347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.837545 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] PATH_INFO: `/v3/projects/94fb247cca724d63bf21a4b138899c27` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:41.838919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fb247cca724d63bf21a4b138899c27)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:41.841798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:41.916529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fb247cca724d63bf21a4b138899c27', 'request_id': 'req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.926644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fb247cca724d63bf21a4b138899c27', 'request_id': 'req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.935023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fb247cca724d63bf21a4b138899c27', 'request_id': 'req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.958035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] The token cache is being invalidate because project 94fb247cca724d63bf21a4b138899c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:28:41.958451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.959187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-20d92709-1c92-41b5-b23b-cb8d8d39b1a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:41.966912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1204/2372] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:28:41 2022] DELETE /identity/v3/projects/94fb247cca724d63bf21a4b138899c27 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:28:41.968155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:41.968817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:41.968817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.013525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:42.014829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:42.097160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1169/2373] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:42.105156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.105502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.105787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.131777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:42.133612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:42.179213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1205/2374] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:42.181253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.202817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47a25a-9d38-4f97-b95c-dbd9eec0932c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.217551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.218200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.218475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.218752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.219896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:28:42.221068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.223232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.245368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.252373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1170/2375] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:28:42.271640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a15406-6712-4553-9cb7-f31c8e6048b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.278689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.280272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.280638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.281151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.281993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.285728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.343003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1206/2376] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:28:42 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:28:42.348201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.364706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c05e-0283-4ef7-883e-344ec2f813ab admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.452791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.453563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.453862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.454267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.455070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.457106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.502275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1171/2377] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:28:42 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:28:42.502627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.522387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409895d7-d5e4-4664-9141-745f13da256a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.535287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.535960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.536223 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.536489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.541696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.544014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.576777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1207/2378] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:28:42.583481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.621667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21924a53-4863-4c9c-91af-5c9769a66fc2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.649355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.649984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.650264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.650540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.651173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.664978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288cfa4-29d9-41b1-a0ab-5de82c198529 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.701880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1172/2379] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:28:42.704801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:42.722687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3660e-c6ed-43ef-a69a-370b37c27342 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:42.734082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:42.735161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.735737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.736170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] PATH_INFO: `/v3/projects/3d49795183aa48c2a9cc86f8ed0da939/users/98c7ef8997494ecf92bb79c51461b064/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.770900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:42.793090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d49795183aa48c2a9cc86f8ed0da939, user_id=98c7ef8997494ecf92bb79c51461b064, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:42.804984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:42.843850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1208/2380] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:28:42 2022] PUT /identity/v3/projects/3d49795183aa48c2a9cc86f8ed0da939/users/98c7ef8997494ecf92bb79c51461b064/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:28:42.856172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f49ed-92ed-4c45-ba73-22f3dc73a2f4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:42.856911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f49ed-92ed-4c45-ba73-22f3dc73a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:42.857336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f49ed-92ed-4c45-ba73-22f3dc73a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:42.902942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a22f49ed-92ed-4c45-ba73-22f3dc73a2f4 None None] MFA Rules not processed for user `98c7ef8997494ecf92bb79c51461b064`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:42.907150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a22f49ed-92ed-4c45-ba73-22f3dc73a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:43.014935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:43.135019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1173/2381] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:43.148652 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:43.150473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:43.224225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5855-901e-4b42-b561-2d053b60be5d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:43.229284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:43.229842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:43.229944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:43.230215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:43.230518 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:43.230961 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:43.231414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:43.246943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:43.249267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bded-aeb1-49da-a9aa-5f85df9c6f71 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:43.257034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:43.258763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:43.262878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1209/2382] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:28:43.389473 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:43.389773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:43.390045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:43.417450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] MFA Rules not processed for user `95558d9a28bf4a0bac9dd72445999781`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:43.417809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1174/2383] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:28:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:28:43.419878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:43.437820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:43.439859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:43.514533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1210/2384] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:28:43.577628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d281c4-88cc-4d6d-b877-72a4d8ee72a9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:43.586489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd485390-ca01-40e2-849c-086969281204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:43.588490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485390-ca01-40e2-849c-086969281204 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:43.588928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485390-ca01-40e2-849c-086969281204 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:43.589253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485390-ca01-40e2-849c-086969281204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:43.602197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd485390-ca01-40e2-849c-086969281204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:43.604060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd485390-ca01-40e2-849c-086969281204 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:43.628894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1175/2385] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 21:28:53.499481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=56,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:53.612845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=72 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:28:56.533115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:56.603827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cda7a-7a46-41a0-80fd-02c25d40b32b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:56.608426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:56.609059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:56.609342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:56.609617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] PATH_INFO: `/v3/users/98c7ef8997494ecf92bb79c51461b064` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:56.611353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c7ef8997494ecf92bb79c51461b064)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:56.613383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:56.660956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c7ef8997494ecf92bb79c51461b064'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:56.661655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c7ef8997494ecf92bb79c51461b064'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:56.669189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c7ef8997494ecf92bb79c51461b064'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:56.682850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c7ef8997494ecf92bb79c51461b064'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:56.683712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c7ef8997494ecf92bb79c51461b064'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:56.697794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1211/2386] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:28:56 2022] DELETE /identity/v3/users/98c7ef8997494ecf92bb79c51461b064 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:28:56.702246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:56.702761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:56.702859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:56.723151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:28:56.724411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:56.791087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1176/2387] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:28:56.800845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:56.802336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:56.865434 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92887f-3fcc-4a16-894d-608afa3090d2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:56.869095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:56.869698 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:56.869976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:56.870268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:56.880642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:56.882075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9898e64-65ef-40c2-9aa7-6f70a6d4ac2b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:56.892097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1212/2388] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:28:57.223177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:28:57.224770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:28:57.238611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57141f-ab41-4797-9624-58ef74959b46 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:28:57.243192 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:28:57.243920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:28:57.244294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:28:57.244630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] PATH_INFO: `/v3/projects/3d49795183aa48c2a9cc86f8ed0da939` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:28:57.245807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d49795183aa48c2a9cc86f8ed0da939)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:28:57.247882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:28:57.298951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d49795183aa48c2a9cc86f8ed0da939', 'request_id': 'req-b2cdc867-4083-462d-a5bf-b3a4dae18350'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:57.307108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d49795183aa48c2a9cc86f8ed0da939', 'request_id': 'req-b2cdc867-4083-462d-a5bf-b3a4dae18350'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:57.316309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d49795183aa48c2a9cc86f8ed0da939', 'request_id': 'req-b2cdc867-4083-462d-a5bf-b3a4dae18350'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:57.328481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] The token cache is being invalidate because project 3d49795183aa48c2a9cc86f8ed0da939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:28:57.329316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:57.330124 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b2cdc867-4083-462d-a5bf-b3a4dae18350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:28:57.331774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1177/2389] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:28:57 2022] DELETE /identity/v3/projects/3d49795183aa48c2a9cc86f8ed0da939 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:29:04.027218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.028085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.028580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.030114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1213/2390] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:29:04.036999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871374-3be5-4d17-ab4d-438ab6329b69 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.037426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871374-3be5-4d17-ab4d-438ab6329b69 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.037859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871374-3be5-4d17-ab4d-438ab6329b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.063713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-35871374-3be5-4d17-ab4d-438ab6329b69 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:04.065932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35871374-3be5-4d17-ab4d-438ab6329b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.149895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1178/2391] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:04.162467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:04.165037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.194344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:04.237457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5db45-cfe3-4e04-926e-6a4fc038f453 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:04.241003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:04.241641 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.242046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.242327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.253435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:04.254854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81af0c-10eb-428f-ad83-b1964fd3e8eb service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:04.265917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1214/2392] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:04.337681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.338064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.338342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.339724 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1179/2393] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:29:04.344459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.344830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.345720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.366372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:04.367614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.388566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:04.390385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.411190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:04.422654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1215/2394] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:04.435079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:04.436899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.448707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb629b2b-6498-4294-95ab-6e2a061cd25c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:04.452372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:04.453005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.453289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.453559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.454384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f6d3-53f1-4b9b-9310-fd3170dde8a5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:04.459191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:04.459823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.460108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.460378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.462077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.462431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:04.463931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46726dc5-6035-4175-a817-3914d16c27a5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:04.472182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1216/2395] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:29:04.526134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:04.527606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423272d-a4a5-41a0-b213-aa293f6b5d8b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:04.537929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1180/2396] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:29:04.559311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.559724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.560089 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.561513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1217/2397] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:29:04.567817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c539f-4d37-4f36-84eb-afc6a4b12b05 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.568106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c539f-4d37-4f36-84eb-afc6a4b12b05 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.568380 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c539f-4d37-4f36-84eb-afc6a4b12b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.588260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0d9c539f-4d37-4f36-84eb-afc6a4b12b05 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:04.589455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0d9c539f-4d37-4f36-84eb-afc6a4b12b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.599458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1181/2398] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:04.610622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:04.613204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:04.628552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a481-2b0d-4d24-95f0-961bfa7eef90 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:04.632296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:04.632912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:04.633193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:04.633468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:04.644960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:04.646420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28eb88-c869-4a9d-a602-b7d71271fb74 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:04.657197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1218/2399] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:10.827209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:10.828362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:10.828362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:10.835250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:10.835757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:10.835865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:10.850526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:10.851775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:10.856537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:10.857759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:10.910313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1182/2400] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:10.913905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1219/2401] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:10.925341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:10.926814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:10.929910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:10.931450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:10.984934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:10.997890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6dc9c-97a4-4d21-ab0b-32732aa079cd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:11.002475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634405f-ac7f-4ab3-8d42-835e866e4141 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:11.003720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:11.004323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:11.004645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:11.004866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:11.007453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:11.008078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:11.008357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:11.008632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:11.009849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:11.014126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:11.015669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2455711-04ce-4e16-9d88-941f04a0e15b service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:11.016769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:11.019064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:11.025877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1183/2402] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 21:29:11.033914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1220/2403] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 21:29:11.787134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a47c-067e-41c9-b065-0cfb134d2970 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:11.787483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a47c-067e-41c9-b065-0cfb134d2970 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:11.787958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a47c-067e-41c9-b065-0cfb134d2970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:11.810337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6744a47c-067e-41c9-b065-0cfb134d2970 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:11.811670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6744a47c-067e-41c9-b065-0cfb134d2970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:11.819922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1184/2404] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:29:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:11.836893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:11.851762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7b6c-2c50-4559-9be7-da8cacbd5c13 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:11.858832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:11.859444 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:11.859762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:11.860051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:11.869162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:11.870861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:11.880749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1221/2405] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:29:12.150465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cd30-4ad6-43db-b42b-e00d8d000cdd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:12.150933 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cd30-4ad6-43db-b42b-e00d8d000cdd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:12.151030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cd30-4ad6-43db-b42b-e00d8d000cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:12.171257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ec32cd30-4ad6-43db-b42b-e00d8d000cdd None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:12.172510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec32cd30-4ad6-43db-b42b-e00d8d000cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:12.180856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1185/2406] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:12.199463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:12.212879 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a321aa8-a6c7-4f27-9ac6-412351fe19eb service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:12.218314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:12.218929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:12.219212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:12.219489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:12.230023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:12.231649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:12.240957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1222/2407] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:29:12.612312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:12.612864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:12.613233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:12.634841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:12.636369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:12.659864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1186/2408] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:12.671934 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:12.684318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d2482-1f51-404f-8afb-994d00fb47ad service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:12.689991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:12.690608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:12.690881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:12.691151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:12.700091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:12.701716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6d3fc-f3df-434d-93af-b9b89b0b779d service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:12.710852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1223/2409] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:29:14.918748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:14.920266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.007052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d8507-a764-43ad-8e4b-91e471483290 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.011956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.012823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.012921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.013189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] PATH_INFO: `/v3/users/95558d9a28bf4a0bac9dd72445999781` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.014372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=95558d9a28bf4a0bac9dd72445999781)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.016317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.069404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95558d9a28bf4a0bac9dd72445999781'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.070257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95558d9a28bf4a0bac9dd72445999781'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.083717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95558d9a28bf4a0bac9dd72445999781'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.103445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95558d9a28bf4a0bac9dd72445999781'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.112216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95558d9a28bf4a0bac9dd72445999781'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.128575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1187/2410] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:29:14 2022] DELETE /identity/v3/users/95558d9a28bf4a0bac9dd72445999781 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:29:15.132816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.133365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.133539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.154412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:15.155527 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.218427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1224/2411] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:29:15.228930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:15.230357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.292644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920902-c6a9-4cfa-bf31-fb4ded5394ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.296885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.297575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.297855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.298205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.305803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:15.308708 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.310720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb94fc-d7ff-4a68-90e7-8c53fb8ab057 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.323365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1188/2412] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:15.506556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:15.508255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.541502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557a31-7625-4675-8f77-8e7d3d7f00ce None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.546546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.547190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.547480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.547765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] PATH_INFO: `/v3/projects/767cd74e36bf4ebb8e30b6f82d2a9164` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.549025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=767cd74e36bf4ebb8e30b6f82d2a9164)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.551243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.610898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767cd74e36bf4ebb8e30b6f82d2a9164', 'request_id': 'req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.618729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767cd74e36bf4ebb8e30b6f82d2a9164', 'request_id': 'req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.624177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767cd74e36bf4ebb8e30b6f82d2a9164', 'request_id': 'req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.636727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] The token cache is being invalidate because project 767cd74e36bf4ebb8e30b6f82d2a9164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:29:15.637020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.637736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39dc40e2-0d6c-425c-86d7-2ea9cf05f9a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:29:15.639245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1225/2413] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:29:15 2022] DELETE /identity/v3/projects/767cd74e36bf4ebb8e30b6f82d2a9164 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:29:15.644727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.645015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.645289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.665863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:15.667170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.729009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1189/2414] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:15.737830 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.738122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.738690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.772344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:15.773566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:15.781556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1226/2415] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:15.786341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:15.796484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d0e0-a803-42cd-aa25-9e40aa23ac7f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.802811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.803587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.803986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.804348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.805127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:29:15.805715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.807417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.821169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1190/2416] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:29:15.825682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:15.835341 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d686d-e12b-405d-8449-10bb9de611a9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.841222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.841885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.842160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.842426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.843057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.844972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.876664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:15.890776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:15.891565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1227/2417] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:29:15 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:29:15.902943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e34fbe-d4c4-4cd6-badb-8bd10200ea99 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:15.966430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:15.967085 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:15.967350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:15.967522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:15.968160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:15.970075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:15.998856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1191/2418] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:29:15 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:29:16.000111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:16.010802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee083ca-8947-4023-b71d-fd6fd8cbf829 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:16.015458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:16.016090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:16.016368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:16.016640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:16.017471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:16.019171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:16.029989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1228/2419] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:29:16.034825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:16.046032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357762f-3cb0-466d-afaf-5eb14da781c2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:16.051499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:16.052133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:16.052474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:16.052749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:16.053393 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:16.054965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f931fc-afac-4a06-9e3b-537cb2e2acb9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:16.067434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1192/2420] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:29:16.072703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:16.084007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1129b4-5865-40c5-a379-c9a367507923 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:16.088692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:16.089351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:16.089630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:16.089901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] PATH_INFO: `/v3/projects/cb5effc588a44c458bd5bab9b1448b87/users/04f32e144d6a4957b8c65befbce1f75a/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:16.105688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5effc588a44c458bd5bab9b1448b87, user_id=04f32e144d6a4957b8c65befbce1f75a, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:16.108356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:16.122700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1229/2421] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:29:16 2022] PUT /identity/v3/projects/cb5effc588a44c458bd5bab9b1448b87/users/04f32e144d6a4957b8c65befbce1f75a/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:29:16.128213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45560-4269-4181-bc38-8a2a834f96e1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:16.128502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45560-4269-4181-bc38-8a2a834f96e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:16.128781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45560-4269-4181-bc38-8a2a834f96e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:16.148832 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bbb45560-4269-4181-bc38-8a2a834f96e1 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:16.149884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bbb45560-4269-4181-bc38-8a2a834f96e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:16.234700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1193/2422] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:16.246346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:16.248127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:16.336670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b89ea-0f4c-4d0f-8a7d-656384bb2125 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:16.340318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:16.340956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:16.341233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:16.341504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:16.352756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:16.354199 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:16.364919 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1230/2423] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:17.097505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89434-6a80-4ebc-aeb4-6fc934725978 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:17.097984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89434-6a80-4ebc-aeb4-6fc934725978 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:17.098102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89434-6a80-4ebc-aeb4-6fc934725978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:17.105849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:17.148917 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05f89434-6a80-4ebc-aeb4-6fc934725978 None None] MFA Rules not processed for user `04f32e144d6a4957b8c65befbce1f75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:17.150408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05f89434-6a80-4ebc-aeb4-6fc934725978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:17.256350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1194/2424] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:17.268391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:17.269908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:17.359855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd92d6-9db2-406c-83f8-c65f29527e1c service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:17.370069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:17.370959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:17.371408 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:17.371878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:17.389317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:17.390854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab6773-9729-49ec-a857-c2a63398b815 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:17.401618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1231/2425] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:29:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:29:18.834776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:18.835362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:18.835362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:18.854683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] MFA Rules not processed for user `04f32e144d6a4957b8c65befbce1f75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:18.856011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:18.865430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1195/2426] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:29:27.397132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:28.849019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=34 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:42.810245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:42.811006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:42.811006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:42.843368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:29:42.845378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:42.906888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1232/2427] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:29:42.930323 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:29:42.931840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:29:43.002189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6f13-c13d-4964-80c1-31ae85cabfd9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:29:43.008114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:29:43.008744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:29:43.009024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:29:43.009292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:29:43.017710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:29:43.019275 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:29:43.027764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1196/2428] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:29:52.898357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:29:53.024652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=32 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:30:25.940455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:25.942625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:26.017170 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69dfb9-90ba-49ef-9b22-65c6db0d61e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:26.023347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:26.023990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:26.024277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:26.024552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:26.026311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:26.087942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:26.089534 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:26.100554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1233/2429] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:26.197804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:26.207075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f111-b824-4b7d-9789-cf791a131ccd service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:26.211163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:26.211802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:26.212078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:26.212345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:26.214154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:26.227186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:26.228618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:26.239922 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1197/2430] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:29.963068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:29.964569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:29.976911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454c7-9891-4918-bb63-ef04d79fc571 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:29.980530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:29.981124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:29.981404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:29.981679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:29.983304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:29.997623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:29.999095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19c36c-835f-4156-9172-a5772b692148 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:30.011306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1234/2431] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:36.234545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:30:40.005424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=62 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:30:46.868617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:46.877821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004731b-c708-40f0-8085-d6fa283612ec service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:46.881472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:46.882060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:46.882333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:46.882600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:46.884361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:46.894508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:46.896028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:46.904617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1198/2432] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:47.020644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:47.022182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:47.022745 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:47.043649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] MFA Rules not processed for user `04f32e144d6a4957b8c65befbce1f75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:30:47.044870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:47.051667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1235/2433] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:30:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:30:47.297486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:47.307572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b0efb-52db-4ede-8237-c0cd55787984 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:47.311263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:47.311896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:47.312175 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:47.312451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:47.314208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:47.324234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:47.325663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0d202-a2c1-49d5-bf59-6cb2a2d06cd9 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:47.334172 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1199/2434] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:47.660278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:30:47.661649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:47.737016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94dc7e-c9a9-451e-905a-632d869baa51 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:30:47.742567 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:30:47.743161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:30:47.743465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:30:47.743765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:30:47.745718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:30:47.757610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:30:47.759168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:30:47.768848 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1236/2435] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:30:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:30:57.330241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:30:57.764731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=35 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:17.483777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:17.484262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:17.484359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:17.503243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] MFA Rules not processed for user `0d773c7f6aca4ea2ad2872f67e9116af`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:31:17.504499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:17.558827 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1200/2436] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:31:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:31:27.551885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=25 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:28.470476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:31:28.483766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c038a-92dc-453a-b98e-1fe443bcff51 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:31:28.489203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:31:28.489808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:28.490097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:28.490359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:28.492945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:28.505296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:31:28.506862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0dee3-5fdf-4b1c-a147-e3254424a681 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:31:28.518073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1237/2437] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:31:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:31:31.673576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:31:31.675003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:31.691897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134685fe-7b4b-475b-9009-a444f3f2d23e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:31:31.697461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:31:31.698068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:31.698353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:31.698625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:31.700396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:31.763310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:31:31.764947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:31:31.774941 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1201/2438] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:31:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:31:38.514196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:41.769873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=28 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:44.517418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:44.517876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:44.517989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:44.539725 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] MFA Rules not processed for user `85a24cb16a864a03920ccb359c13ec79`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:31:44.540937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:44.547940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1238/2439] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:31:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:31:44.874423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:31:44.883729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877015d-d92d-4b5e-897c-d0bd8d5879c1 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:31:44.887280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:31:44.887964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:44.888186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:44.888457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:44.890158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:44.900118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:31:44.901530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:31:44.909809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1202/2440] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:31:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:31:54.533631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:54.905938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:31:59.327882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:31:59.342733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a740d-6088-4a9b-8e10-209077c58a84 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:31:59.348343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:31:59.348952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:31:59.349228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:31:59.349499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:31:59.351214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:31:59.364181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:31:59.365757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:31:59.376747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1239/2441] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:31:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:01.023659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.025078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:01.042263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dcfa0-f6bd-4834-b0f5-bb09a07aba81 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.047235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.047867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.048151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.048451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.050183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:01.063167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.064786 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.076981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1203/2442] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:01.087017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.106926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c7f21-56a1-4408-804e-f8874600c416 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.112020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.112979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.113662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.113844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] PATH_INFO: `/v3/users/04f32e144d6a4957b8c65befbce1f75a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.115768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f32e144d6a4957b8c65befbce1f75a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.119191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.168213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f32e144d6a4957b8c65befbce1f75a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.168860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f32e144d6a4957b8c65befbce1f75a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.176413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f32e144d6a4957b8c65befbce1f75a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.188557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f32e144d6a4957b8c65befbce1f75a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.189264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e8268986-27b3-49f8-9277-c6f9f425cf5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f32e144d6a4957b8c65befbce1f75a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.202280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1240/2443] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:32:01 2022] DELETE /identity/v3/users/04f32e144d6a4957b8c65befbce1f75a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:32:01.396075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.409328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365d2e-3f3c-4508-96a6-cc80ea9816e2 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.473967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.474853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.475131 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.475402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] PATH_INFO: `/v3/projects/cb5effc588a44c458bd5bab9b1448b87` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.476572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5effc588a44c458bd5bab9b1448b87)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.478461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.532199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5effc588a44c458bd5bab9b1448b87', 'request_id': 'req-116294c7-9428-427b-9f10-f95caf7a0d06'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.549185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5effc588a44c458bd5bab9b1448b87', 'request_id': 'req-116294c7-9428-427b-9f10-f95caf7a0d06'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.557550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5effc588a44c458bd5bab9b1448b87', 'request_id': 'req-116294c7-9428-427b-9f10-f95caf7a0d06'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.568158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] The token cache is being invalidate because project cb5effc588a44c458bd5bab9b1448b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:32:01.568443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.569084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-116294c7-9428-427b-9f10-f95caf7a0d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:01.570579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1204/2444] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:32:01 2022] DELETE /identity/v3/projects/cb5effc588a44c458bd5bab9b1448b87 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:32:01.575997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.576315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.576594 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.597009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:01.598296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:01.666543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1241/2445] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:01.674301 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.674589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.674865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.695048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:01.696325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:01.704841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1205/2446] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:01.709013 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.719394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54e4a9-59e0-45be-8b63-e6c5a5e0cf60 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.724005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.724604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.724883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.725157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.725803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:32:01.726132 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.727762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.739782 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1242/2447] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:32:01.744136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.755883 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300827a3-2091-466d-956d-80bdcfcc2ede None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.761535 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.762167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.762444 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.762705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.763356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.766460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.799258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1206/2448] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:32:01 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:32:01.802945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.813603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90f69-5377-4308-aac5-958ca270f908 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.860819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:01.884691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.886069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.886447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.886790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.887649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.889540 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.919196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1243/2449] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:32:01 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:32:01.920706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.931273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002accf6-793d-4428-8c53-9115e122c237 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.936526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.937122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.937401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.937671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.938306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.940108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.950775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1207/2450] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:32:01.954857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:01.965993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abb947-4f67-4333-b448-c84e0d33833c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:01.971232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:01.972016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:01.972299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:01.972575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:01.973243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:01.974836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf872ad3-fc09-4ea6-a7e8-e6cd4916ac6e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:01.986572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1244/2451] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:32:01.990742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:02.000696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7a2a3-0dd2-4bd9-80b3-eb87a4e8d413 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:02.005571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:02.006222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:02.006503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:02.006777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] PATH_INFO: `/v3/projects/8dd167e547564847b4fb509124293162/users/f857517cade84f5fb7a884121351dfa3/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:02.022208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd167e547564847b4fb509124293162, user_id=f857517cade84f5fb7a884121351dfa3, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:02.024956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:02.039348 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1208/2452] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:32:01 2022] PUT /identity/v3/projects/8dd167e547564847b4fb509124293162/users/f857517cade84f5fb7a884121351dfa3/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:32:02.049664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:02.049954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:02.050230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:02.080228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] MFA Rules not processed for user `f857517cade84f5fb7a884121351dfa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:02.081469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:02.155009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1245/2453] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:32:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:02.162894 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:02.164659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:02.179637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:02.227474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bcf83-2456-4188-9d28-95f7ebab53ff admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:02.231814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:02.232428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:02.232709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:02.232982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:02.241847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:02.243273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:02.253098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1209/2454] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:02.677232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:02.678816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:02.755743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcafa63-209c-4a16-a21b-34cbc38ea657 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:02.761556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:02.762165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:02.762442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:02.762720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:02.764449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:02.777235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:02.778831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae983e9e-8a8f-47c6-9555-535cd5333881 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:02.790618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1246/2455] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:12.248802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=20 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:12.785521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:24.737478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:24.803990 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc636469-350b-44da-aa22-715ba63fdc5a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:24.814624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:24.815274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:24.815577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:24.815855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] PATH_INFO: `/v3/users/f857517cade84f5fb7a884121351dfa3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:24.816994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f857517cade84f5fb7a884121351dfa3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:24.818929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:24.868024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f857517cade84f5fb7a884121351dfa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:24.868993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f857517cade84f5fb7a884121351dfa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:24.876658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f857517cade84f5fb7a884121351dfa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:24.888865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f857517cade84f5fb7a884121351dfa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:24.889696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f857517cade84f5fb7a884121351dfa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:24.902900 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1210/2456] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:32:24 2022] DELETE /identity/v3/users/f857517cade84f5fb7a884121351dfa3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:32:24.907625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:24.908129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:24.908230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:24.928898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:24.930149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:25.000973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1247/2457] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:25.010638 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:25.012283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:25.071815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e51c-e2f4-4ff8-be72-e8f709b76c95 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:25.075633 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:25.076236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:25.076513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:25.076809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:25.086160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:25.087630 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e023544-d5f7-4c5d-be78-bfa47d7c698d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:25.098418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1211/2458] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:32:25.623684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:25.629537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:25.642292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ca64-1a64-41a6-9a3b-3f0ea8aa87ff None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:25.647111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:25.647789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:25.648085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:25.648328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] PATH_INFO: `/v3/projects/8dd167e547564847b4fb509124293162` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:25.649443 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd167e547564847b4fb509124293162)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:25.651342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:25.712455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd167e547564847b4fb509124293162', 'request_id': 'req-00a68f62-4a69-4d80-bde8-14a83501998d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:25.720692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd167e547564847b4fb509124293162', 'request_id': 'req-00a68f62-4a69-4d80-bde8-14a83501998d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:25.726279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd167e547564847b4fb509124293162', 'request_id': 'req-00a68f62-4a69-4d80-bde8-14a83501998d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:25.738290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] The token cache is being invalidate because project 8dd167e547564847b4fb509124293162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:32:25.738809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:25.739323 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-00a68f62-4a69-4d80-bde8-14a83501998d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:32:25.740900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1248/2459] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:32:25 2022] DELETE /identity/v3/projects/8dd167e547564847b4fb509124293162 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:32:27.264555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f7729-aead-4aaf-a9a5-67b3ea34da0c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:27.265072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f7729-aead-4aaf-a9a5-67b3ea34da0c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:27.265169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f7729-aead-4aaf-a9a5-67b3ea34da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:27.286017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-219f7729-aead-4aaf-a9a5-67b3ea34da0c None None] MFA Rules not processed for user `8c37b14f268f4180a618aa8eeefb5bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:27.287244 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-219f7729-aead-4aaf-a9a5-67b3ea34da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:27.304025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:27.344886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1212/2460] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:32.691585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:32.692165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:32.692282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:32.693806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1249/2461] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:32:32.700834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1eef2-7ea7-43a2-ac0e-c283ab99a953 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:32.701330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1eef2-7ea7-43a2-ac0e-c283ab99a953 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:32.701436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1eef2-7ea7-43a2-ac0e-c283ab99a953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:32.721388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dcf1eef2-7ea7-43a2-ac0e-c283ab99a953 None None] MFA Rules not processed for user `be348541540b47a8acc55e04bbf6adf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:32.722601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dcf1eef2-7ea7-43a2-ac0e-c283ab99a953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:32.775180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1213/2462] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:32:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:32.787599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:32.789551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:32.802880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925422a9-d60d-40b0-b8f0-af705b1f287e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:32.806479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:32.807080 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:32.807365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:32.807672 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:32.815585 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:32.817032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d48ef-48e0-4e2e-a87d-566bce9ea818 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:32.825807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1250/2463] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:32.911563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fe2a-d0c8-4255-8e28-a2cd30651548 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:32.911936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fe2a-d0c8-4255-8e28-a2cd30651548 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:32.912299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fe2a-d0c8-4255-8e28-a2cd30651548 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:32.913771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1214/2464] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 21:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 21:32:32.920177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:32.920462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:32.920761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:32.942210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] MFA Rules not processed for user `dac985d2461d4132af309ded8c7ce1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:32:32.943442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:33.016095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1251/2465] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 21:32:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:32:33.026889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36fe2a-d0c8-4255-8e28-a2cd30651548 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:33.038912 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36fe2a-d0c8-4255-8e28-a2cd30651548 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:33.042530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:33.043757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:33.044035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:33.044307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:33.055137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:33.056616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:33.068341 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1215/2466] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:43.005543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:43.063245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=44 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:32:48.434542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:32:48.436044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:48.492001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634db82-b541-4848-9e29-40fd377ab131 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:32:48.495493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:32:48.496125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:32:48.496406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:32:48.496682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:32:48.498923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:32:48.557360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:32:48.558806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:32:48.570842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1252/2467] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:32:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:32:58.564695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=48 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:07.571662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:07.573073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:07.585089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044facbd-ca15-4b2a-a019-a3812c459a98 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:07.588753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:07.589344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:07.589623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:07.589895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:07.591560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:07.659245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:07.660703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:07.670878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1216/2468] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:10.973181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:10.974733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:10.990107 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb008-28a9-464e-b181-cb7bcbe81309 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:10.995217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:10.996005 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:10.996452 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:10.996729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] PATH_INFO: `/v3/users/0d773c7f6aca4ea2ad2872f67e9116af` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:10.997909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d773c7f6aca4ea2ad2872f67e9116af)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.000131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.051308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d773c7f6aca4ea2ad2872f67e9116af'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.051991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d773c7f6aca4ea2ad2872f67e9116af'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.059746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d773c7f6aca4ea2ad2872f67e9116af'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.074014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d773c7f6aca4ea2ad2872f67e9116af'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.074724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-33c6debe-636b-4603-8c74-9e5e11807558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d773c7f6aca4ea2ad2872f67e9116af'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.088960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1253/2469] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:10 2022] DELETE /identity/v3/users/0d773c7f6aca4ea2ad2872f67e9116af => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:11.273212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.283673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec63b-0fc4-4f7e-b2b7-3d3fc37204fa service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.340779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.341405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.341772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.342071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] PATH_INFO: `/v3/projects/5472cdfe59524ac5b826fba54758c68f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.343270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5472cdfe59524ac5b826fba54758c68f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.345230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.391283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5472cdfe59524ac5b826fba54758c68f', 'request_id': 'req-e211aa6d-6204-4262-a016-5f11dee282a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.398420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5472cdfe59524ac5b826fba54758c68f', 'request_id': 'req-e211aa6d-6204-4262-a016-5f11dee282a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.405582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5472cdfe59524ac5b826fba54758c68f', 'request_id': 'req-e211aa6d-6204-4262-a016-5f11dee282a8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.415982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] The token cache is being invalidate because project 5472cdfe59524ac5b826fba54758c68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:11.416267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.416954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e211aa6d-6204-4262-a016-5f11dee282a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:11.418484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1217/2470] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:11 2022] DELETE /identity/v3/projects/5472cdfe59524ac5b826fba54758c68f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:11.424475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.424762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.425036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.445253 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:11.446418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:11.506302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1254/2471] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:11.514993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.515281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.515575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.535087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:11.536309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:11.544087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1218/2472] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:33:11.548819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.558577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442fb69-cca8-4137-8f28-3b234a15ed12 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.563835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.564429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.564703 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.564974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.565621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:33:11.565921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.567502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.578860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1255/2473] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:33:11.583411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.592817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b95199-1176-453d-b4df-9733c30e016c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.597390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.597975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.598254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.598523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.599145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.601056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.618240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:11.635550 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1219/2474] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:11 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:33:11.639727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.649762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e113282-058d-4fcd-bea9-a108b268d4cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.705178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.705698 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.705973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.706242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.706872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.708815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.717165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:11.730945 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1256/2475] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:11 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:11.736263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.746560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8dce8-5b9f-435c-8c70-3ffd3a841329 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.752049 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.752658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.752939 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.753263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.753881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.755662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.766611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1220/2476] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:11.773009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.784189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7506a2e-0a67-446d-9bde-9d805cc7a980 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.788970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.789566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.789843 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.790110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.790749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.792372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d62af4-df4f-45b2-98fc-8f56d023ef19 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.804396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1257/2477] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:11.808613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:11.818688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad56f9-7b73-451e-bce5-06008b32071b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:11.823198 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:11.823872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.824148 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.824416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] PATH_INFO: `/v3/projects/11ee81c6183143f1ac150a1c6329d135/users/0e1d48d19259466392fc530617eb11bf/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:11.961474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ee81c6183143f1ac150a1c6329d135, user_id=0e1d48d19259466392fc530617eb11bf, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:11.964046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:11.977305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1221/2478] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:11 2022] PUT /identity/v3/projects/11ee81c6183143f1ac150a1c6329d135/users/0e1d48d19259466392fc530617eb11bf/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:11.987856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:11.988322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:11.988582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:12.016412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] MFA Rules not processed for user `0e1d48d19259466392fc530617eb11bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:12.017614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:12.081256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1258/2479] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:12.092543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:12.094277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:12.167903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0e3ce-9fb7-474d-8226-7b80db674373 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:12.173911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:12.174508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:12.174784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:12.175054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:12.182685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:12.184293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724012e9-311f-42c3-bd56-7860e2b5d6a5 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:12.196305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1222/2480] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:14.882734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:14.984769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841c14d-8a34-48b1-9d17-2844a17de3bf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:14.989255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:14.989876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:14.990153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:14.990420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] PATH_INFO: `/v3/users/0e1d48d19259466392fc530617eb11bf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:14.991583 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1d48d19259466392fc530617eb11bf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:14.993716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:15.024240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:15.043956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1d48d19259466392fc530617eb11bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.044734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1d48d19259466392fc530617eb11bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.052553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1d48d19259466392fc530617eb11bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.064936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1d48d19259466392fc530617eb11bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.065592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1d48d19259466392fc530617eb11bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.079218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1259/2481] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:14 2022] DELETE /identity/v3/users/0e1d48d19259466392fc530617eb11bf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:15.083740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.084223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.084319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.134096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:15.135336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:15.193934 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1223/2482] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:15.203713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:15.205138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:15.277115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70d14f-e959-473e-91ff-73dfa201c122 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:15.280937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:15.281542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.281822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.282087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.290787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:15.292238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94474ecc-04bc-4228-93b0-b4f0d9f11e29 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:15.302181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1260/2483] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:15.572668 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:15.574133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:15.586339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3c1fc-474b-404d-8b08-e199491290cb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:15.590940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:15.592179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.592460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.592730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] PATH_INFO: `/v3/projects/11ee81c6183143f1ac150a1c6329d135` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.593888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ee81c6183143f1ac150a1c6329d135)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:15.595828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:15.613062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:15.644554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ee81c6183143f1ac150a1c6329d135', 'request_id': 'req-c4a60b4d-2c71-4efc-a021-c312ef116360'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.651916 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ee81c6183143f1ac150a1c6329d135', 'request_id': 'req-c4a60b4d-2c71-4efc-a021-c312ef116360'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.659173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ee81c6183143f1ac150a1c6329d135', 'request_id': 'req-c4a60b4d-2c71-4efc-a021-c312ef116360'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.669470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] The token cache is being invalidate because project 11ee81c6183143f1ac150a1c6329d135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:15.670463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.671116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c4a60b4d-2c71-4efc-a021-c312ef116360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:15.672612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1224/2484] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:15 2022] DELETE /identity/v3/projects/11ee81c6183143f1ac150a1c6329d135 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:15.677453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.677741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.678018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.726380 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:15.727604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:15.788168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1261/2485] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:15.796775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.797003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.797281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.816635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:15.821025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:15.830029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1225/2486] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:33:15.834358 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:15.844624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30db54e-428d-48ce-b755-777e878bab24 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:15.849187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:15.849780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.850056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.850327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.850976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:33:15.851274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:15.852905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:15.855308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:15.866804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1262/2487] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:33:15.870744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:15.879829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50371cc2-023d-481d-aa64-3646eab38947 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:15.884726 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:15.885326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.885585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.885849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.886475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:15.888948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:15.916347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1226/2488] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:15 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:15.920790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:15.931021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00f6dd-fcf9-4144-8988-607a317b5205 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:15.986714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:15.987318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:15.987635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:15.987912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:15.988553 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:15.990531 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.011154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1263/2489] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:15 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:16.016152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.026177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad582bc-40f8-4c1d-82d8-e3ce78083301 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.031735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.032320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.032618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.032880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.033508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.035063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.044747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1227/2490] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:16.048651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.058519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c342b-7fee-4e76-ad5c-f5891bbf6617 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.063256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.063873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.064152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.064449 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.065109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.066677 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6005adf-0d15-4d75-b8b3-f17075a9ac64 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.086834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1264/2491] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:33:16.088871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.100407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166600f9-8a8a-4cff-af71-a3a514104c2d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.105605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.106257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.106533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.106802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] PATH_INFO: `/v3/projects/a728e33e13014ac896ad9d271d215028/users/86b441bdae974211a9255f42cfae65e6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.121495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a728e33e13014ac896ad9d271d215028, user_id=86b441bdae974211a9255f42cfae65e6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.124631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.139853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1228/2492] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:16 2022] PUT /identity/v3/projects/a728e33e13014ac896ad9d271d215028/users/86b441bdae974211a9255f42cfae65e6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:16.151270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.151612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.151921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.179767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] MFA Rules not processed for user `86b441bdae974211a9255f42cfae65e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:16.180956 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:16.243044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1265/2493] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:16.248075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.308902 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb517e-590f-44a0-b7ab-3723c1aef6e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.315801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.316411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.316699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.316969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.317609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.319491 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.346658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1229/2494] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:16 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:16.350603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.361701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81a084-f7c9-45bf-80cd-6ac34b0951a5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.427657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.428269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.428550 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.428823 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.429457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.431376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.441877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:16.456352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1266/2495] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:16 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:33:16.460339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.470803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56162bca-4336-4116-84e4-20d1c3ee4fde admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.475390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.476025 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.476303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.476573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.477212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.478813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.489329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1230/2496] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:16.493544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.510409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36261f-82fc-43e5-aa31-469809e733f9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.516841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.517521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.517802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.518078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] PATH_INFO: `/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459/users/661b72978bad4775a8d393485a095949/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.534636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da62ba36c2b4a5fad1c29bf6017d459, user_id=661b72978bad4775a8d393485a095949, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.537324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.552400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1267/2497] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:16 2022] PUT /identity/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459/users/661b72978bad4775a8d393485a095949/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:33:16.556446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.566536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448c7c-106b-4145-ae7c-37c6a13eb77f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.576514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:16.633457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.634164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.634516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.634887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.635640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.637325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e5200-36b6-4251-aeae-9a6b16c8dd83 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.648689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1231/2498] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:16.652878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.664115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b0258-2429-4b1d-9664-3d6569a42b21 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.668707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.669350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.669834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.670143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] PATH_INFO: `/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459/users/661b72978bad4775a8d393485a095949/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.672073 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da62ba36c2b4a5fad1c29bf6017d459, user_id=661b72978bad4775a8d393485a095949, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.674684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.684953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1268/2499] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:16 2022] PUT /identity/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459/users/661b72978bad4775a8d393485a095949/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:16.695186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920321db-3810-4114-aa84-539b67423f34 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.695747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920321db-3810-4114-aa84-539b67423f34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.696019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920321db-3810-4114-aa84-539b67423f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.725492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-920321db-3810-4114-aa84-539b67423f34 None None] MFA Rules not processed for user `661b72978bad4775a8d393485a095949`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:16.726700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-920321db-3810-4114-aa84-539b67423f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:16.795253 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1232/2500] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:16.806413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.808148 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:16.865384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ad49e-b410-4342-b4d8-8959222906ee admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.869008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.869609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.869890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.870167 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:16.879416 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:16.880875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:16.890947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1269/2501] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:16.936574 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920321db-3810-4114-aa84-539b67423f34 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:16.937950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-920321db-3810-4114-aa84-539b67423f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:16.949656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920321db-3810-4114-aa84-539b67423f34 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:16.954016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:16.954605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:16.954881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:16.955152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:17.009764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:17.011177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74171a-09ca-4a46-b906-49aa6e1113fd service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:17.020543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1233/2502] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:20.344205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:20.407276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c832b-67d8-4eee-84b5-e23009ca4c2a service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:20.411810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:20.412463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:20.412760 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:20.413014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] PATH_INFO: `/v3/users/86b441bdae974211a9255f42cfae65e6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:20.414145 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b441bdae974211a9255f42cfae65e6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:20.416093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:20.460662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b441bdae974211a9255f42cfae65e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:20.461316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b441bdae974211a9255f42cfae65e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:20.469133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b441bdae974211a9255f42cfae65e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:20.473513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:20.482521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b441bdae974211a9255f42cfae65e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:20.483251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b441bdae974211a9255f42cfae65e6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:20.497036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1270/2503] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:20 2022] DELETE /identity/v3/users/86b441bdae974211a9255f42cfae65e6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:20.501534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:20.502051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:20.502193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:20.522138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:20.523320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:20.552126 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:20.583515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1234/2504] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:20.593859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:20.595243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:20.652053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ad35-4507-4bee-b425-8642502ca877 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:20.656768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:20.657420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:20.657643 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:20.657912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:20.666701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:20.668159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:20.677526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1271/2505] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:20.938100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:20.939637 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:20.953063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3d54b-889f-45dd-9b02-dd4ccf9208c4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:20.958024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:20.958749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:20.959084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:20.959343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] PATH_INFO: `/v3/projects/a728e33e13014ac896ad9d271d215028` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:20.960536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a728e33e13014ac896ad9d271d215028)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:20.962456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.009489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a728e33e13014ac896ad9d271d215028', 'request_id': 'req-47ce8e90-8dda-4112-99ea-e07e9d068a57'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.018087 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a728e33e13014ac896ad9d271d215028', 'request_id': 'req-47ce8e90-8dda-4112-99ea-e07e9d068a57'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.025167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a728e33e13014ac896ad9d271d215028', 'request_id': 'req-47ce8e90-8dda-4112-99ea-e07e9d068a57'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.038023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] The token cache is being invalidate because project a728e33e13014ac896ad9d271d215028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:21.038316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.038969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.040485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1235/2506] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:20 2022] DELETE /identity/v3/projects/a728e33e13014ac896ad9d271d215028 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:21.044513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:21.054994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4344c-486d-41a8-9cfe-c7318c531b1a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:21.115627 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:21.116260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.116541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.116814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] PATH_INFO: `/v3/users/661b72978bad4775a8d393485a095949` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.117946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661b72978bad4775a8d393485a095949)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:21.119913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.172070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661b72978bad4775a8d393485a095949'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.172732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661b72978bad4775a8d393485a095949'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.179822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661b72978bad4775a8d393485a095949'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.192357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661b72978bad4775a8d393485a095949'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.192944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2658d6c4-cd8e-4162-b052-4fa219594db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661b72978bad4775a8d393485a095949'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.204797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:21.208682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1272/2507] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:21 2022] DELETE /identity/v3/users/661b72978bad4775a8d393485a095949 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:21.536299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:21.537801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:21.553808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8e90-8dda-4112-99ea-e07e9d068a57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:21.613035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:21.613654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.613929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.614196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] PATH_INFO: `/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.615294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da62ba36c2b4a5fad1c29bf6017d459)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:21.617195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.663707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da62ba36c2b4a5fad1c29bf6017d459', 'request_id': 'req-e2888358-02ba-4af6-b781-29c880a47a9f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.670921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da62ba36c2b4a5fad1c29bf6017d459', 'request_id': 'req-e2888358-02ba-4af6-b781-29c880a47a9f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.672835 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:21.677442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da62ba36c2b4a5fad1c29bf6017d459', 'request_id': 'req-e2888358-02ba-4af6-b781-29c880a47a9f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.689434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] The token cache is being invalidate because project 2da62ba36c2b4a5fad1c29bf6017d459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:21.689718 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.690339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e2888358-02ba-4af6-b781-29c880a47a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:21.691826 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1236/2508] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:21 2022] DELETE /identity/v3/projects/2da62ba36c2b4a5fad1c29bf6017d459 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:21.696687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.697160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.697255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.717498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:21.718690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:21.776463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1273/2509] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:21.784723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.785011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.785282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.804665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:21.805848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:21.814544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1237/2510] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:21.818610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:21.830284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5568d5-47de-4473-9cd2-b085bad338a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:21.836017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:21.836635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.836918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.837192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.837846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:33:21.838151 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:21.839791 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.852059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1274/2511] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:33:21.856395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:21.866230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7835b-359f-49c7-ad93-e74cd23b74d7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:21.870802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:21.871397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.871719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.872007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.873276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:21.875235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.901475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1238/2512] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:21 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:21.906003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:21.915924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6f7e-0dfa-4ec8-a704-5ade5e14f245 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:21.971744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:21.972277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:21.972569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:21.972851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:21.973493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:21.975438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:21.996436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1275/2513] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:21 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:22.001196 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.011462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb3459-c381-40cc-87cb-bba12abfb7ee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.016112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.016697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.016976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.017246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.017881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.019459 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.029170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1239/2514] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:22.033334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.043135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c04257-58d8-4bae-a89d-a40c483bbe32 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.048225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.048807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.049083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.049352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.050014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.051636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849d22-7867-4feb-9d75-de33f494c8f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.063581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1276/2515] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:22.067695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.077799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8b1c8-3c8e-444d-aa45-89a438ee4da2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.082340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.082978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.083280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.083580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] PATH_INFO: `/v3/projects/f868325583154784a59479c364042a90/users/93274aaf6b9045958fe2d800cd534ecc/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.098027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f868325583154784a59479c364042a90, user_id=93274aaf6b9045958fe2d800cd534ecc, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.100833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.113810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1240/2516] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:22 2022] PUT /identity/v3/projects/f868325583154784a59479c364042a90/users/93274aaf6b9045958fe2d800cd534ecc/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:22.125717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.125998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.126286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.153673 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] MFA Rules not processed for user `93274aaf6b9045958fe2d800cd534ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:22.154994 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:22.210016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:22.220700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1277/2517] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:22.225199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.293135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6331ea-51d0-40d9-a737-9c67a3bba0f3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.298604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.299188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.299468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.299769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.300405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.302282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.328628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1241/2518] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:22 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:22.332742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.344001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf5a03-a98c-4017-a775-b948bb1cec92 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.403570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.404361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.404640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.404913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.405557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.407789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.427890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1278/2519] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:22 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:22.431928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.442921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27893-84c1-4e4c-9120-a937d655fa6a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.447505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.448132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.448413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.448685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.449351 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.451675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.462059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1242/2520] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:22.466191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.477042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf230d-46f1-4471-b11a-153cd6b66548 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.481668 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.482316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.482592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.482888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] PATH_INFO: `/v3/projects/84c9f968fbbf491ebbaba33915be975b/users/66ffad9ed8054918bdec60a8e53d449c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.500555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c9f968fbbf491ebbaba33915be975b, user_id=66ffad9ed8054918bdec60a8e53d449c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.502962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.516901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1279/2521] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:22 2022] PUT /identity/v3/projects/84c9f968fbbf491ebbaba33915be975b/users/66ffad9ed8054918bdec60a8e53d449c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:22.521078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.532340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e66992-fadb-4dbb-aa43-924b6b3efb4a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.586188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:22.593378 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.594213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.594733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.595231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.596340 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.598184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93a657-7e0c-405e-8be8-72c4e01bdce8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.609113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1243/2522] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:22.613938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.624767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931d0-4faa-41f3-be0c-57e37af663fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.629607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.630435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.630620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.631328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] PATH_INFO: `/v3/projects/84c9f968fbbf491ebbaba33915be975b/users/66ffad9ed8054918bdec60a8e53d449c/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.633333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c9f968fbbf491ebbaba33915be975b, user_id=66ffad9ed8054918bdec60a8e53d449c, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.636091 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.647693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1280/2523] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:22 2022] PUT /identity/v3/projects/84c9f968fbbf491ebbaba33915be975b/users/66ffad9ed8054918bdec60a8e53d449c/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:22.658839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.659117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.659388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.686613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] MFA Rules not processed for user `66ffad9ed8054918bdec60a8e53d449c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:22.687860 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:22.754777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1244/2524] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:22.766329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:22.768074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:22.839084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7d7d-2306-4183-b81b-a6c98dbedb38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:22.843739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:22.844377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:22.844658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:22.844931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:22.855116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:22.856622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969958-51a1-4d96-b4de-5f67d11ebc34 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:22.868260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1281/2525] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:22.943644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:23.005073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fd0a3-17c1-4c9f-a6d5-62e56837c9cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:23.009602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:23.010223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:23.010504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:23.010779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] PATH_INFO: `/v3/users/93274aaf6b9045958fe2d800cd534ecc` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:23.011942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93274aaf6b9045958fe2d800cd534ecc)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:23.014862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:23.060318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93274aaf6b9045958fe2d800cd534ecc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.061026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93274aaf6b9045958fe2d800cd534ecc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.069673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93274aaf6b9045958fe2d800cd534ecc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.082236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93274aaf6b9045958fe2d800cd534ecc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.082942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93274aaf6b9045958fe2d800cd534ecc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.091165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:23.097425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1245/2526] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:22 2022] DELETE /identity/v3/users/93274aaf6b9045958fe2d800cd534ecc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:23.102261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:23.102751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:23.103102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:23.122026 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:23.124057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:23.169263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:23.185367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1282/2527] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:23.195586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:23.196995 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:23.253372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8702e-153f-4617-a13b-8978b3cbcf5d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:23.257295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:23.257851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:23.258150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:23.258421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:23.268399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:23.269801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:23.279270 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1246/2528] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:23.551296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:23.552831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:23.565432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2c921-089c-4e27-9a10-443793fdb006 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:23.570063 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:23.570699 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:23.570979 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:23.571732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] PATH_INFO: `/v3/projects/f868325583154784a59479c364042a90` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:23.572537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f868325583154784a59479c364042a90)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:23.574948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:23.622910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f868325583154784a59479c364042a90', 'request_id': 'req-a5d0c223-eb29-49bb-894a-c498132949b3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.632833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f868325583154784a59479c364042a90', 'request_id': 'req-a5d0c223-eb29-49bb-894a-c498132949b3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.639656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f868325583154784a59479c364042a90', 'request_id': 'req-a5d0c223-eb29-49bb-894a-c498132949b3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.650595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] The token cache is being invalidate because project f868325583154784a59479c364042a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:23.650880 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.651513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.652991 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1283/2529] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:23 2022] DELETE /identity/v3/projects/f868325583154784a59479c364042a90 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:23.657955 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:23.668041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c3808-ca7c-4e8c-b7b6-0deb9d7748ac service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:23.726706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:23.727440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:23.727857 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:23.728194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] PATH_INFO: `/v3/users/66ffad9ed8054918bdec60a8e53d449c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:23.729406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ffad9ed8054918bdec60a8e53d449c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:23.731452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:23.779833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ffad9ed8054918bdec60a8e53d449c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.780479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ffad9ed8054918bdec60a8e53d449c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.787473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ffad9ed8054918bdec60a8e53d449c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.799570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ffad9ed8054918bdec60a8e53d449c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.800182 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2dd378e1-7ec1-4558-b25b-9d9e02c36906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ffad9ed8054918bdec60a8e53d449c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:23.815660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1247/2530] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:23 2022] DELETE /identity/v3/users/66ffad9ed8054918bdec60a8e53d449c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:24.090587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.092264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:24.105826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0c223-eb29-49bb-894a-c498132949b3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.164321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.164951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.165224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.165491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] PATH_INFO: `/v3/projects/84c9f968fbbf491ebbaba33915be975b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.166593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c9f968fbbf491ebbaba33915be975b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.168530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.214448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c9f968fbbf491ebbaba33915be975b', 'request_id': 'req-ee2d6576-68a9-4c22-bf8d-7009f7485470'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:24.222326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c9f968fbbf491ebbaba33915be975b', 'request_id': 'req-ee2d6576-68a9-4c22-bf8d-7009f7485470'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:24.228849 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c9f968fbbf491ebbaba33915be975b', 'request_id': 'req-ee2d6576-68a9-4c22-bf8d-7009f7485470'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:24.240401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] The token cache is being invalidate because project 84c9f968fbbf491ebbaba33915be975b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:24.240684 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:24.241333 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ee2d6576-68a9-4c22-bf8d-7009f7485470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:24.242774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1284/2531] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:24 2022] DELETE /identity/v3/projects/84c9f968fbbf491ebbaba33915be975b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:24.248694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.249006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.249286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.269799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:24.271005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:24.284881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:24.336414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1248/2532] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:24.345702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.345990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.346300 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.355101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:24.369220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:24.370797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:24.378962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1285/2533] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:24.383274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.394223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707dcd-d6c2-4c05-90c3-69c625c1ed33 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.399211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.399864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.400141 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.400411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.401060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:33:24.401357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.402935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.415479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1249/2534] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:33:24.419896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.429121 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d854a-4610-4353-99c4-33f38fd1afd3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.433815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.434419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.435230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.435513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.436202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.438090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.470303 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1286/2535] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:24.471453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.481984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659ad66-9c03-4c7d-ba90-09723662cf33 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.542352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.542958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.543234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.543501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.544149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.546397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.569012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1250/2536] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:33:24.572710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.584075 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49367c5c-f111-469c-b3c9-7e9e73c4cff0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.588987 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.589537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.589813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.590084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.590729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.592406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.603031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1287/2537] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:24.607983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.617951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62871822-28ed-4840-8c70-8e395767e853 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.624943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.625556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.625829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.626090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.626737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.628394 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29383971-0310-46fb-b6fb-4fb9b9b12bfb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.641063 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1251/2538] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:24.645241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.655232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a7715-7fcf-4e85-8cf0-516731f27812 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.660207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.660852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.661126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.661392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] PATH_INFO: `/v3/projects/965cbcceb4d544ad8007916f784823b8/users/490f09b0ade9440391e3d25f157da8b0/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.676069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965cbcceb4d544ad8007916f784823b8, user_id=490f09b0ade9440391e3d25f157da8b0, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.678682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.692750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1288/2539] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:24 2022] PUT /identity/v3/projects/965cbcceb4d544ad8007916f784823b8/users/490f09b0ade9440391e3d25f157da8b0/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:24.703706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.704162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.704258 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.733821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] MFA Rules not processed for user `490f09b0ade9440391e3d25f157da8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:24.735056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:24.800589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1252/2540] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:33:24.805271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.870015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e4fc-3b5c-4fa5-b671-ec5d38d056b3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.875483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.876570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.876570 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.876837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.877532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.879679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:24.911259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1289/2541] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:24.912441 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:24.923610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feb649-c908-4f6f-bd5f-9c067fbc6d7c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:24.936931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:24.987816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:24.988430 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:24.988710 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:24.988991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:24.989634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:24.991566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.017631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1253/2542] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:24 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:33:25.021192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.032498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f844f71-571d-42ca-8cbe-d74a4e620479 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.037382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.038054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.038396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.038731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.039435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.041138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.053342 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1290/2543] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:25.057595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.068770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda52e1-7be0-4e50-85fc-09a363a999a4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.073527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.074177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.074458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.074728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] PATH_INFO: `/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a/users/218118f414b54190a4ebd17b730513cd/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.093603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d63c8cc69941fcb32fe3ae7648694a, user_id=218118f414b54190a4ebd17b730513cd, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.097009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.112304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1254/2544] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:25 2022] PUT /identity/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a/users/218118f414b54190a4ebd17b730513cd/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:33:25.116454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.126877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133fe0a-6f46-4c40-88de-de5ef737f27c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.183367 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.183999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.184284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.184561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.185940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.187557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3155a-3f2f-4b2d-acad-95e12b0cb986 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.197234 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1291/2545] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:25.202083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.212365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae047af2-aa82-4e6e-9fed-ff023a85c07f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.216972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.217611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.217885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.218151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] PATH_INFO: `/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a/users/218118f414b54190a4ebd17b730513cd/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.220120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d63c8cc69941fcb32fe3ae7648694a, user_id=218118f414b54190a4ebd17b730513cd, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.222704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.232714 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1255/2546] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:25 2022] PUT /identity/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a/users/218118f414b54190a4ebd17b730513cd/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:25.242719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.243009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.243290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.264560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:25.272789 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] MFA Rules not processed for user `218118f414b54190a4ebd17b730513cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:25.273999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:25.342344 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1292/2547] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:25.352896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.354606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:25.414928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3b9e-4e9b-4c91-b2ba-ce2788503f55 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.418566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.419209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.419461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.419753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.430108 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.431553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.442354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1256/2548] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:25.454986 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.456410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:25.496069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:25.498023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:25.509391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311396f9-f338-4720-a963-73ebacd7da1a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.513971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.514561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.514842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.515116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.561674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481e7e-f2b7-4a07-8f6f-b5a8be9d94f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:25.567406 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:25.568038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:25.568321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:25.568593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:25.570301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:25.573474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.574930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.582832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:25.584438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb246f-15d0-421f-bc24-41fbe2ce6f97 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:25.585052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1257/2549] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:33:25.594827 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1293/2550] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:28.879327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:28.889584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:28.942665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe496f9-673b-4869-8923-10f3530e1df0 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:28.947137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:28.947782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:28.948067 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:28.948389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] PATH_INFO: `/v3/users/490f09b0ade9440391e3d25f157da8b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:28.949450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490f09b0ade9440391e3d25f157da8b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:28.952356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:28.997694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490f09b0ade9440391e3d25f157da8b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:28.998336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490f09b0ade9440391e3d25f157da8b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.005344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490f09b0ade9440391e3d25f157da8b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.018231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490f09b0ade9440391e3d25f157da8b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.018878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490f09b0ade9440391e3d25f157da8b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.032583 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1258/2551] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:28 2022] DELETE /identity/v3/users/490f09b0ade9440391e3d25f157da8b0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:29.037074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:29.037542 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:29.037657 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:29.056710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:29.057887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:29.126175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1294/2552] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:29.136018 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:29.137456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:29.195626 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e5fe4-1755-47b6-af51-9b3ad2ce2f34 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:29.199437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:29.200142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:29.200391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:29.200655 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:29.209598 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:29.211011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:29.220846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1259/2553] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:29.482438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:29.484039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:29.496502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3931bc-88fd-4fc8-97cb-35870da63dea None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:29.501847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:29.502593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:29.503501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:29.503501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] PATH_INFO: `/v3/projects/965cbcceb4d544ad8007916f784823b8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:29.504625 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=965cbcceb4d544ad8007916f784823b8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:29.507055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:29.509225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:29.556009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965cbcceb4d544ad8007916f784823b8', 'request_id': 'req-8a605429-2be4-4bff-882f-ffe69b9684ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.563269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965cbcceb4d544ad8007916f784823b8', 'request_id': 'req-8a605429-2be4-4bff-882f-ffe69b9684ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.569071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965cbcceb4d544ad8007916f784823b8', 'request_id': 'req-8a605429-2be4-4bff-882f-ffe69b9684ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.580795 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] The token cache is being invalidate because project 965cbcceb4d544ad8007916f784823b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:29.581081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.581728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.583194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1295/2554] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:29 2022] DELETE /identity/v3/projects/965cbcceb4d544ad8007916f784823b8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:29.587267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:29.597764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6f734-0b0f-417e-b456-a259a4b1bc6d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:29.628654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:29.658529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:29.659103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:29.659383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:29.659689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] PATH_INFO: `/v3/users/218118f414b54190a4ebd17b730513cd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:29.660813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] RBAC: Authorizing `identity:delete_user(user_id=218118f414b54190a4ebd17b730513cd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:29.662697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:29.708315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218118f414b54190a4ebd17b730513cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.708970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218118f414b54190a4ebd17b730513cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.715919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218118f414b54190a4ebd17b730513cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.727963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218118f414b54190a4ebd17b730513cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.728678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-51a9a632-6935-48a1-aa06-347aab6cdbde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218118f414b54190a4ebd17b730513cd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:29.742434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1260/2555] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:33:29 2022] DELETE /identity/v3/users/218118f414b54190a4ebd17b730513cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:30.014629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.016211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:30.029915 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a605429-2be4-4bff-882f-ffe69b9684ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.095422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.096160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.096370 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.096639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] PATH_INFO: `/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.097962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d63c8cc69941fcb32fe3ae7648694a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.100331 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.146744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d63c8cc69941fcb32fe3ae7648694a', 'request_id': 'req-146b4379-a6ef-47c1-9105-bee7ec459d3d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:30.154692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d63c8cc69941fcb32fe3ae7648694a', 'request_id': 'req-146b4379-a6ef-47c1-9105-bee7ec459d3d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:30.160548 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d63c8cc69941fcb32fe3ae7648694a', 'request_id': 'req-146b4379-a6ef-47c1-9105-bee7ec459d3d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:30.172110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] The token cache is being invalidate because project d7d63c8cc69941fcb32fe3ae7648694a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:33:30.172394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:30.173025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-146b4379-a6ef-47c1-9105-bee7ec459d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:33:30.174482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1296/2556] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:33:30 2022] DELETE /identity/v3/projects/d7d63c8cc69941fcb32fe3ae7648694a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:33:30.179299 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.179845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.179964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.199423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:30.201203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:30.257863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1261/2557] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:30.320909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.321195 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.321470 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.342592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:30.343838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:30.351526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1297/2558] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:30.355858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.365493 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee12510-f5f9-4b30-be97-0ea7d46fc328 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.370016 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.370609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.370889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.371162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.371843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:33:30.372274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.373866 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.385157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1262/2559] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:33:30.389946 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.398974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd82fb-23e5-43ae-be9c-fb5fbd1300ba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.404429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.405056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.405339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.405868 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.406506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.408748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.420477 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:30.438954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1298/2560] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:33:30 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:30.443399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.454349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd48c68-fd91-447f-b5bd-b1c9ac2b35d1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.486684 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:30.511064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.511694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.511975 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.512247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.513797 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.515817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.536886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1263/2561] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:33:30 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:33:30.541087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.551654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00c1c3-8cac-4364-97ab-b47277af0d4e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.556193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.556786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.557069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.557343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.558006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.560269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.570829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1299/2562] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:30.574891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.585466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03b3f9-8b4d-4d39-a0bc-571d6b2cdf73 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.589892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.590472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.590751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.591015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.591670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.593226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f206d-b309-4704-a7fc-982cbf85a173 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.603251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1264/2563] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:33:30.607645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.617438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b6eb2-a448-4f12-83ea-9137dd01fce8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.622573 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.623455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.623787 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.624066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] PATH_INFO: `/v3/projects/df5de0620d2b4dfc85507dfb42e6fbd2/users/e31c2192953042359a08ba6c67dfa65d/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.639717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5de0620d2b4dfc85507dfb42e6fbd2, user_id=e31c2192953042359a08ba6c67dfa65d, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.642381 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.655753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1300/2564] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:33:30 2022] PUT /identity/v3/projects/df5de0620d2b4dfc85507dfb42e6fbd2/users/e31c2192953042359a08ba6c67dfa65d/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:33:30.665592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cbe51-acba-4e53-af85-d7a3d744f600 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.665872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cbe51-acba-4e53-af85-d7a3d744f600 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.666160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cbe51-acba-4e53-af85-d7a3d744f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.693748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a67cbe51-acba-4e53-af85-d7a3d744f600 None None] MFA Rules not processed for user `e31c2192953042359a08ba6c67dfa65d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:30.694940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a67cbe51-acba-4e53-af85-d7a3d744f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:30.756978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1265/2565] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:30.769317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:30.770992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:30.826785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914404-ae19-406d-825f-da179f9fafbb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:30.830423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:30.831017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:30.831298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:30.831597 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:30.840029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:30.841476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:30.850539 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1301/2566] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:33:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:33:40.754032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:40.846851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:44.216812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:44.217355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:44.217355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:44.248103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:33:44.249300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:44.321751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1266/2567] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:33:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:33:44.369898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:33:44.371377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:33:44.445373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f434576-403c-46a2-ba1a-4adda9555131 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:33:44.450635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:33:44.451317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:33:44.451640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:33:44.451914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:33:44.461249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:33:44.462832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:33:44.471747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1302/2568] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:33:54.314504 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=32 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:33:54.467575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=32 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:04.117489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:04.127471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f510ec6-38ac-4efc-b2e3-adad3f37de92 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:04.131208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:04.132307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:04.132657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:04.132999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:04.134806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:04.229553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:04.230981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:04.251821 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1267/2569] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:04.318020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:04.319431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:04.394700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a912d4d-07ec-43d3-aa25-6afab1c1e56e service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:04.398168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:04.398753 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:04.399027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:04.399297 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:04.401017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:04.411254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:04.412721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6b945-30c3-414d-a376-3babfd5bf68a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:04.420856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1303/2570] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:04.493590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:04.495059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:04.506809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792cd-cd26-4ddf-a155-924c06da1cc1 service glance] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:04.510354 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:04.510946 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:04.511225 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:04.511496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:04.513220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:04.527766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:04.529186 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:04.541306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1268/2571] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:06.319771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:06.320278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:06.320563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:06.340561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:06.342062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:06.349273 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1304/2572] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:34:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:06.361231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:06.372345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d9fab-68db-4e41-b6f4-05a35c08f09f service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:06.377988 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:06.378577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:06.378854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:06.379127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:06.387291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:06.388948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54283e61-7835-434d-a973-08a9633c0109 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:06.397772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1269/2573] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:34:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:34:16.335774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=38 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:16.394540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=55 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:18.333867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:18.395359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe7e88-be9d-496d-8989-5b37d636f6f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:18.399921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:18.400537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:18.400822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:18.401095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] PATH_INFO: `/v3/users/e31c2192953042359a08ba6c67dfa65d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:18.402221 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31c2192953042359a08ba6c67dfa65d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:18.404160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:18.450000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31c2192953042359a08ba6c67dfa65d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:18.450645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31c2192953042359a08ba6c67dfa65d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:18.457993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31c2192953042359a08ba6c67dfa65d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:18.469860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31c2192953042359a08ba6c67dfa65d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:18.471114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31c2192953042359a08ba6c67dfa65d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:18.484307 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1305/2574] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:34:18 2022] DELETE /identity/v3/users/e31c2192953042359a08ba6c67dfa65d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:18.489792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:18.490303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:18.490446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:18.510333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:18.511524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:18.577387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1270/2575] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:18.587691 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:18.589117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:18.648439 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edf57-284b-4d39-8c2c-5da1248dd869 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:18.652112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:18.652718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:18.653001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:18.653277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:18.663240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:18.664717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbacba2-7d9d-4f8a-89b7-f8ac8268c089 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:18.674081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1306/2576] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:18.993304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:18.994750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:19.006624 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659b53a-15ab-45a4-815b-0831a9686958 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.011312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.011978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.012257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.012532 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] PATH_INFO: `/v3/projects/df5de0620d2b4dfc85507dfb42e6fbd2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.013954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5de0620d2b4dfc85507dfb42e6fbd2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.015877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.063908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5de0620d2b4dfc85507dfb42e6fbd2', 'request_id': 'req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:19.071593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5de0620d2b4dfc85507dfb42e6fbd2', 'request_id': 'req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:19.078019 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5de0620d2b4dfc85507dfb42e6fbd2', 'request_id': 'req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:19.089297 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] The token cache is being invalidate because project df5de0620d2b4dfc85507dfb42e6fbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:34:19.089518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:19.090143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5a2349f0-9278-4c3e-a795-7b40b6d2e12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:19.091689 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1271/2577] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:34:18 2022] DELETE /identity/v3/projects/df5de0620d2b4dfc85507dfb42e6fbd2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:19.096700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.096990 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.097294 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.117975 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:19.119185 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:19.136171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:19.180972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1307/2578] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:19.190879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.191179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.191436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.211174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:19.212397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:19.220374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1272/2579] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:19.225109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.234601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1409b7-33d7-4826-9c44-7e4dfa5820a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.240407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.240992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.241267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.241535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.242174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:34:19.242471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.244108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.255426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1308/2580] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:34:19.259723 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.269472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b4768-0019-43c4-a236-8a31ac2deb1b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.274075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.274918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.275201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.275472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.276146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.278029 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.310711 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1273/2581] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:34:19 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:19.314158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.324328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9464d-e44c-4f66-a913-3b5eb0d4c0da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.380275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.380909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.381064 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.381359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.381997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.383974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.405386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1309/2582] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:34:19 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:19.410241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.420541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc8778-d77c-4703-b3ee-e6de684b2506 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.425421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.426012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.426293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.426566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.427204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.428877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.438523 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1274/2583] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:19.442350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.452225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160151-84f3-4c55-a0f4-6d49c20dbbd8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.457653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.458249 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.458529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.458803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.459471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.461117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10c877-8e79-4558-8e08-7caaf6d8b373 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.472577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1310/2584] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:19.476704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.489783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026196-75f5-4bf2-9454-3aab7a5a7a0e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.495027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.495722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.496000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.496272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] PATH_INFO: `/v3/projects/b9a36910972a4d87b508b918af36bbd5/users/691a25c97f4748119366c35dd4688765/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.511184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a36910972a4d87b508b918af36bbd5, user_id=691a25c97f4748119366c35dd4688765, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.513843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.527303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1275/2585] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:19 2022] PUT /identity/v3/projects/b9a36910972a4d87b508b918af36bbd5/users/691a25c97f4748119366c35dd4688765/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:19.532984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4247aa-e6e3-4bde-af3d-a56cde3d7725 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.533916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4247aa-e6e3-4bde-af3d-a56cde3d7725 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.534417 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4247aa-e6e3-4bde-af3d-a56cde3d7725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.553957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8e4247aa-e6e3-4bde-af3d-a56cde3d7725 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:19.555160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8e4247aa-e6e3-4bde-af3d-a56cde3d7725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:19.615147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1311/2586] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:19.625071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:19.626815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:19.649742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:19.686570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c79a6d-92ad-4675-b77b-2f8ba4df2382 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:19.690165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:19.690748 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:19.691022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:19.691290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:19.700007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:19.701405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:19.711026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1276/2587] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:34:23.836071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:23.836622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:23.836622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:23.869453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] MFA Rules not processed for user `691a25c97f4748119366c35dd4688765`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:23.870437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:23.905269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:23.961759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1312/2588] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:23.966227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:23.976373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba01b8-ceb1-4055-ab15-aea5f28cf8ba service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:23.981127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:23.981991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:23.982344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:23.982687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:23.983436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:23.985544 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.014422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1277/2589] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:34:23 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:24.019306 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:24.034041 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2d77-4912-4f8b-8f21-f668d27e6e3a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:24.100251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:24.101023 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:24.101376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:24.101660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:24.102305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:24.104447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.132093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1313/2590] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:34:24 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:24.133460 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:24.145984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84896733-be79-41fc-a89f-62e1cddea3f8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:24.150607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:24.151236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:24.151511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:24.152056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:24.152838 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:24.154581 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.166260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1278/2591] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:24.170776 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:24.184289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbde1-4572-4ed2-8599-5f83363ab499 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:24.190681 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:24.191368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:24.191828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:24.192138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] PATH_INFO: `/v3/projects/cd154266453440d9b564748a7f371c63/users/bb350f44508d427a8b2355822785b8c6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:24.209941 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd154266453440d9b564748a7f371c63, user_id=bb350f44508d427a8b2355822785b8c6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:24.212761 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.244412 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1314/2592] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:24 2022] PUT /identity/v3/projects/cd154266453440d9b564748a7f371c63/users/bb350f44508d427a8b2355822785b8c6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:24.248561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:24.262904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691a48-d12f-42b9-a169-497816bee9b3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:24.344178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:24.344619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:24.344907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:24.345177 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:24.345977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:24.347601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003311c-4ca6-4394-910a-bb3230e09443 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.360357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1279/2593] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:34:24.364841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:24.378867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde07a68-9bb9-4f0e-9d55-cb50a591fee4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:24.383812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:24.384489 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:24.384779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:24.385051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] PATH_INFO: `/v3/projects/cd154266453440d9b564748a7f371c63/users/bb350f44508d427a8b2355822785b8c6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:24.387130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd154266453440d9b564748a7f371c63, user_id=bb350f44508d427a8b2355822785b8c6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:24.390407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:24.402501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1315/2594] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:24 2022] PUT /identity/v3/projects/cd154266453440d9b564748a7f371c63/users/bb350f44508d427a8b2355822785b8c6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:34:28.448241 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a616-7860-4ecb-9347-897e224788dc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:28.448740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a616-7860-4ecb-9347-897e224788dc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:28.448740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a616-7860-4ecb-9347-897e224788dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:28.494510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6763a616-7860-4ecb-9347-897e224788dc None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:28.496059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6763a616-7860-4ecb-9347-897e224788dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:28.613114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1280/2595] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:28.633486 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:28.636088 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:28.760360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc927279-1e6d-48d3-afa9-ca224f393710 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:28.770592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:28.771228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:28.771512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:28.771816 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:28.786152 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:28.787954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eedb0-53e1-47ff-ae3d-8fb08854c09c service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:28.796493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:28.803992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1316/2596] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:34:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:34:29.015585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763a616-7860-4ecb-9347-897e224788dc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:29.100214 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:29.131483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763a616-7860-4ecb-9347-897e224788dc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:29.136075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:29.136842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:29.137811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:29.138267 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:29.140364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:29.238853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:29.241044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:29.271565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1281/2597] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:30.280220 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:30.281320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:30.281992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:30.326316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] MFA Rules not processed for user `bb350f44508d427a8b2355822785b8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:30.328481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:30.442505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1317/2598] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:30.449220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:30.450752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:30.589093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5069-1f93-498d-9179-5e3c8f38a017 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:30.600853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:30.603257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:30.603561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:30.603998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:30.708363 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:30.711652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:30.732046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1282/2599] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:34.875999 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:34.878318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:34.898272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2414b6-558a-424d-a1d8-07eea38495c0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:34.901971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:34.902580 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:34.902854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:34.903127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:34.905209 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:34.977628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:34.979090 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:34.990912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1318/2600] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:40.719773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:41.833528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:41.835798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:41.913226 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:41.915350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:41.932964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda1324-bfdb-4570-97fc-270daee29fe3 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:41.935015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bec9-ca16-47c6-a35a-6314933dd381 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:41.937713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:41.939575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:41.939856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:41.940135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] PATH_INFO: `/v3/users/469561e39c274301b16677d12cac3a67` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:41.945289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469561e39c274301b16677d12cac3a67)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:41.946512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:41.947211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:41.947337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:41.947660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:41.948012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:41.949863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:41.963788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:41.965412 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:41.976034 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1283/2601] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:42.009947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469561e39c274301b16677d12cac3a67'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.010592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469561e39c274301b16677d12cac3a67'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.020012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469561e39c274301b16677d12cac3a67'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.033168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469561e39c274301b16677d12cac3a67'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.033881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469561e39c274301b16677d12cac3a67'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.038696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:42.053960 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1319/2602] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:34:41 2022] DELETE /identity/v3/users/469561e39c274301b16677d12cac3a67 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:42.228976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:42.245147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbe79-48b6-4312-8ab4-444328b75aff service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:42.319631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:42.321342 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:42.321461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:42.321696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] PATH_INFO: `/v3/projects/e46e2167ee15481fab472e5b101c9c60` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:42.323076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46e2167ee15481fab472e5b101c9c60)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:42.325115 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:42.394778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46e2167ee15481fab472e5b101c9c60', 'request_id': 'req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.404193 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46e2167ee15481fab472e5b101c9c60', 'request_id': 'req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.413560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46e2167ee15481fab472e5b101c9c60', 'request_id': 'req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.425778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] The token cache is being invalidate because project e46e2167ee15481fab472e5b101c9c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:34:42.426098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.426768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.428291 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1284/2603] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:34:42 2022] DELETE /identity/v3/projects/e46e2167ee15481fab472e5b101c9c60 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:42.432364 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:42.433833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:42.450436 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06589d0-e4d2-4d2b-aea4-a662b1f315e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:42.517716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:42.518346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:42.518621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:42.518973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] PATH_INFO: `/v3/users/d6ffc0b8319b47b8b96fb6d591097963` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:42.520244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ffc0b8319b47b8b96fb6d591097963)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:42.522260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:42.566971 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ffc0b8319b47b8b96fb6d591097963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.567645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ffc0b8319b47b8b96fb6d591097963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.574729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ffc0b8319b47b8b96fb6d591097963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.586160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ffc0b8319b47b8b96fb6d591097963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.586783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-73788fea-c78e-4ee7-8766-15c04d477103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ffc0b8319b47b8b96fb6d591097963'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.600537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1320/2604] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:34:42 2022] DELETE /identity/v3/users/d6ffc0b8319b47b8b96fb6d591097963 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:42.779059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:42.780809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:42.794227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c85e5-1eb4-4952-8c3b-ef1e51567b1b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:42.811119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:42.854405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:42.855062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:42.855287 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:42.855577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] PATH_INFO: `/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:42.856699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d750c69f7c364c5ab2f5d2541b8430e4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:42.858575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:42.904566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd750c69f7c364c5ab2f5d2541b8430e4', 'request_id': 'req-bdf44747-9cc9-4f16-ade6-6967e07531f5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.912197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd750c69f7c364c5ab2f5d2541b8430e4', 'request_id': 'req-bdf44747-9cc9-4f16-ade6-6967e07531f5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.918536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd750c69f7c364c5ab2f5d2541b8430e4', 'request_id': 'req-bdf44747-9cc9-4f16-ade6-6967e07531f5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.929242 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] The token cache is being invalidate because project d750c69f7c364c5ab2f5d2541b8430e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:34:42.929535 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.930226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:42.931737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1285/2605] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:34:42 2022] DELETE /identity/v3/projects/d750c69f7c364c5ab2f5d2541b8430e4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:52.596924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=55 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:52.926188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:54.199165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:54.199727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:54.199831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:54.220257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] MFA Rules not processed for user `691a25c97f4748119366c35dd4688765`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:54.221920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:54.277836 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1321/2606] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:54.971752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:54.973274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:55.038741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf44747-9cc9-4f16-ade6-6967e07531f5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:55.042508 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:55.043117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:55.043398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:55.043720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:55.045510 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:55.110970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:55.112350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:55.121888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1286/2607] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:34:58.323423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:58.326238 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:58.366030 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb4ce4-6df6-49e9-96bb-da02f67bd565 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:58.371660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:58.372404 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:58.372748 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:58.373421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] PATH_INFO: `/v3/users/8c37b14f268f4180a618aa8eeefb5bc6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:58.375019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c37b14f268f4180a618aa8eeefb5bc6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:58.377308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:58.454889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c37b14f268f4180a618aa8eeefb5bc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.455892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c37b14f268f4180a618aa8eeefb5bc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.465421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c37b14f268f4180a618aa8eeefb5bc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.482277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c37b14f268f4180a618aa8eeefb5bc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.483353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0c0f8cdc-c516-4358-8225-a293b5bb8796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c37b14f268f4180a618aa8eeefb5bc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.505105 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1322/2608] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:34:58 2022] DELETE /identity/v3/users/8c37b14f268f4180a618aa8eeefb5bc6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:58.682296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:58.693041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366ba55-8290-4679-8f39-d7974c39e240 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:58.749738 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:58.750319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:58.750603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:58.750874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] PATH_INFO: `/v3/projects/bc8ed5f414b947e2ab1815f5a775a2c5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:58.752044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8ed5f414b947e2ab1815f5a775a2c5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:58.753919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:58.803502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8ed5f414b947e2ab1815f5a775a2c5', 'request_id': 'req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.805986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:58.810929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8ed5f414b947e2ab1815f5a775a2c5', 'request_id': 'req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.819552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8ed5f414b947e2ab1815f5a775a2c5', 'request_id': 'req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.830507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] The token cache is being invalidate because project bc8ed5f414b947e2ab1815f5a775a2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:34:58.830796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.831435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bcb55f5-2e05-4240-8ace-93f9bc5c00de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:34:58.832943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1287/2609] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:34:58 2022] DELETE /identity/v3/projects/bc8ed5f414b947e2ab1815f5a775a2c5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:58.837954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:58.838240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:58.838515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:58.858079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:58.859263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:58.918335 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1323/2610] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:58.926489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:58.926780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:58.927058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:58.948461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:58.949670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:58.958501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1288/2611] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:58.962642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:58.972495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0911a-d3bf-415e-bf7c-7f88f6b965e9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:58.977074 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:58.977692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:58.977957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:58.978263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:58.978911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:34:58.979218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:58.980867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:58.992372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1324/2612] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:34:58.996842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.007040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d6f7-ede3-4dde-bc4a-f7804d95dc37 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.011644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.012260 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.012536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.012803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.013433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.015303 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.042264 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1289/2613] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:34:58 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:59.046600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.052974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:59.057021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99b76-9710-4bfa-ac70-03935bc436ce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.116802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.117407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.117688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.117963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.118603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.120587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.148356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1325/2614] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:34:59 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:59.152698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.163484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401b457-40e6-4af4-bd48-2fedde96925b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.168107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.168725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.169007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.169284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.169925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.171500 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.182471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1290/2615] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:34:59.186495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.197378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add09b06-b7a1-4a46-a60c-089ceddaad4d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.202366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.202969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.203308 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.203867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.204549 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.206196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d066cd6-eda9-4a96-92de-1d2f55266fb9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.217884 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1326/2616] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:59.222152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.231950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a689-790e-4ea1-8414-908dea938ca7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.237075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.237709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.237980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.238247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] PATH_INFO: `/v3/projects/52cfd94fb4fc485a8b20f0ab01e5326c/users/c0206bc23ca64f2791bf885198e907d8/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.253305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52cfd94fb4fc485a8b20f0ab01e5326c, user_id=c0206bc23ca64f2791bf885198e907d8, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.256311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.269424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1291/2617] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:59 2022] PUT /identity/v3/projects/52cfd94fb4fc485a8b20f0ab01e5326c/users/c0206bc23ca64f2791bf885198e907d8/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:59.282279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.282561 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.282841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.310311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] MFA Rules not processed for user `c0206bc23ca64f2791bf885198e907d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:59.311522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:59.376496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1327/2618] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:59.380903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.440792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272876-4631-4d9e-8f9b-fbb025161d5f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.445336 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.445926 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.446202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.446470 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.447095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.448996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.478433 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1292/2619] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:34:59 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:34:59.482728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.495622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bd263-80de-4a60-9a96-324c89011aff None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.560819 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.561435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.561712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.561982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.562676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.564784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.586541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1328/2620] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:34:59 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:34:59.590923 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.602073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07891a0-67e4-4fef-a432-6f9279159e8d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.606982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.607604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.607884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.608153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.608791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.611138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.622307 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1293/2621] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:59.627268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.632757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:59.640899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a307bc8-44ba-4fba-95d7-b821860061fb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.646520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.647180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.647460 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.647772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] PATH_INFO: `/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299/users/11652709d35f4b3fb965c5860545dd04/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.664029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c57d38a190a4b8fb5b29becf0d0a299, user_id=11652709d35f4b3fb965c5860545dd04, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.666666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.680959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1329/2622] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:59 2022] PUT /identity/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299/users/11652709d35f4b3fb965c5860545dd04/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:59.684984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.695194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da93552-2e71-4dce-8625-dd1500e57fe2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.744294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:34:59.756359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.758057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.758057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.758057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.758679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.760328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa8d9-338a-425c-8e72-099c478297da admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.771229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1294/2623] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:34:59.775504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.786878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a8d3-2cce-4d71-8c0a-c392083c81e7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:34:59.791860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:34:59.792556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.792964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.793248 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] PATH_INFO: `/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299/users/11652709d35f4b3fb965c5860545dd04/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.795343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c57d38a190a4b8fb5b29becf0d0a299, user_id=11652709d35f4b3fb965c5860545dd04, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:34:59.798693 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:34:59.809608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1330/2624] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:34:59 2022] PUT /identity/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299/users/11652709d35f4b3fb965c5860545dd04/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:34:59.820928 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:34:59.821217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:34:59.821589 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:34:59.850227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] MFA Rules not processed for user `11652709d35f4b3fb965c5860545dd04`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:34:59.851492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:34:59.921467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1295/2625] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:34:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:34:59.932866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:34:59.934654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:00.013133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae95cd-33f9-4540-83ed-71cfce9cbd47 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:00.016861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:00.017456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:00.017732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:00.018001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:00.028213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:00.030110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a93f8-6ead-4651-bb5c-8859d3e5a6e0 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:00.040285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1331/2626] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:34:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:00.189061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:00.252090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a246-4d75-414c-8ac1-e4182290bc92 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:00.257518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:00.257518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:00.257782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:00.257879 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] PATH_INFO: `/v3/users/c0206bc23ca64f2791bf885198e907d8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:00.259036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0206bc23ca64f2791bf885198e907d8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:00.260971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:00.318334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0206bc23ca64f2791bf885198e907d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.318905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0206bc23ca64f2791bf885198e907d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.326492 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0206bc23ca64f2791bf885198e907d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.340410 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0206bc23ca64f2791bf885198e907d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.341185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0206bc23ca64f2791bf885198e907d8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.343095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:00.363845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1296/2627] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:00 2022] DELETE /identity/v3/users/c0206bc23ca64f2791bf885198e907d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:00.365933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:00.365933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:00.366246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:00.385572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:00.386837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:00.447472 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1332/2628] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:00.457495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:00.458925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:00.525517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ff482-6d11-4092-90f0-cf0e175c687c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:00.529967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:00.530111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:00.530327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:00.530594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:00.540599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:00.542028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:00.552608 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1297/2629] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:00.859104 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:00.860928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:00.886779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0210b72-961c-468c-acbd-cf896b751730 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:00.893372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:00.894085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:00.894371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:00.894645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] PATH_INFO: `/v3/projects/52cfd94fb4fc485a8b20f0ab01e5326c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:00.897271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=52cfd94fb4fc485a8b20f0ab01e5326c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:00.899207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:00.950914 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:00.956694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52cfd94fb4fc485a8b20f0ab01e5326c', 'request_id': 'req-d1193a08-1307-4504-a08a-7c4f912afafc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.967392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52cfd94fb4fc485a8b20f0ab01e5326c', 'request_id': 'req-d1193a08-1307-4504-a08a-7c4f912afafc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.974457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52cfd94fb4fc485a8b20f0ab01e5326c', 'request_id': 'req-d1193a08-1307-4504-a08a-7c4f912afafc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.987989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] The token cache is being invalidate because project 52cfd94fb4fc485a8b20f0ab01e5326c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:00.988345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.989664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:00.994187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1333/2630] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:00 2022] DELETE /identity/v3/projects/52cfd94fb4fc485a8b20f0ab01e5326c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:00.995580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:01.008825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19f09e-131d-4324-aeb8-084d7d884e0d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:01.095391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:01.096138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:01.096685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:01.097201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] PATH_INFO: `/v3/users/11652709d35f4b3fb965c5860545dd04` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:01.098677 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=11652709d35f4b3fb965c5860545dd04)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:01.100789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:01.171075 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11652709d35f4b3fb965c5860545dd04'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.172230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11652709d35f4b3fb965c5860545dd04'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.181530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11652709d35f4b3fb965c5860545dd04'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.196642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11652709d35f4b3fb965c5860545dd04'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.197740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-febe109c-e0e6-4e67-bf1c-21319ec1fbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11652709d35f4b3fb965c5860545dd04'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.213573 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1298/2631] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:00 2022] DELETE /identity/v3/users/11652709d35f4b3fb965c5860545dd04 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:01.675019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:01.676662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:01.690842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1193a08-1307-4504-a08a-7c4f912afafc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:01.768502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:01.769265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:01.769605 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:01.769944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] PATH_INFO: `/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:01.771134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c57d38a190a4b8fb5b29becf0d0a299)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:01.773119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:01.827855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c57d38a190a4b8fb5b29becf0d0a299', 'request_id': 'req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.844885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c57d38a190a4b8fb5b29becf0d0a299', 'request_id': 'req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.853937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c57d38a190a4b8fb5b29becf0d0a299', 'request_id': 'req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.867400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] The token cache is being invalidate because project 1c57d38a190a4b8fb5b29becf0d0a299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:01.867721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.868386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-733dcd27-cfd2-4bb0-9a79-3a2f703b1ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:01.870094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1334/2632] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:01 2022] DELETE /identity/v3/projects/1c57d38a190a4b8fb5b29becf0d0a299 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:01.875981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:01.876352 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:01.876829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:01.913479 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:01.915121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:01.927862 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:01.989023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1299/2633] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:01.997034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:01.997407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:01.997759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.021181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:02.022523 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:02.031235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1335/2634] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:02.035468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.045789 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5133-dade-45ce-8cc0-ceb81bf6a4a6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.050585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.051211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.051490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.051795 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.052453 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:02.052754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.054381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.068165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1300/2635] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:02.072496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.082649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1349d0-b3d7-4cb4-b7b8-52d23ab6fb2d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.087974 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.088595 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.088872 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.089142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.089783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.091714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.121717 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1336/2636] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:02.126805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.137163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90926e-d731-4215-a3de-269f024a47fb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.197572 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.198136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.198461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.198683 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.199317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.201262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.229001 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1301/2637] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:02.230385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.242034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4df4fd-1e85-4446-9666-aa9d411f1d10 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.247094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.247878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.248458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.248831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.249557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.252018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.263130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1337/2638] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:02.267746 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.277722 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac0dd-4122-498e-be78-d975bc429c43 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.283249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.283873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.284149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.284417 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.285052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.286610 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51d1b-579c-4485-b725-0aac31c3e3e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.297591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1302/2639] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:02.301806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.312275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196f6ca-dbb5-4c55-a8c8-ed9332494aa2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.317072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.317726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.318001 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.318270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] PATH_INFO: `/v3/projects/08340ef11a7149ed9bc3bb84c68bc15c/users/8415b27415d24df1ab6f0e2cdabe070a/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.334175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08340ef11a7149ed9bc3bb84c68bc15c, user_id=8415b27415d24df1ab6f0e2cdabe070a, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.336858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.345822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:02.358285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1338/2640] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:02 2022] PUT /identity/v3/projects/08340ef11a7149ed9bc3bb84c68bc15c/users/8415b27415d24df1ab6f0e2cdabe070a/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:35:02.362871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.363155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.363429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.391434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] MFA Rules not processed for user `8415b27415d24df1ab6f0e2cdabe070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:02.392782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:02.458605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1303/2641] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:02.463244 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.525883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023914-2347-4c36-a0da-a59ffc631539 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.532111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.532730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.533006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.533287 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.533921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.535877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.564762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1339/2642] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:02.568911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.580325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071521cc-69b0-4713-aa2d-f76f3f9471f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.585048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:02.639568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.640203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.640507 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.640790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.641426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.643333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.666715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1304/2643] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:02.670706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.681694 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba4c0-cb91-460d-92b5-e2ed67716d04 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.686825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.687578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.687869 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.688235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.688893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.690494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.701204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1340/2644] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:02.705334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.715304 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b13c3-1945-46cb-af58-ea5ea7bffefc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.720464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.721100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.721407 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.721681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] PATH_INFO: `/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61/users/a6b8c49ae8304c748e6217e3fbfdbf75/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.736753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6013de7cea4a9db7e237f7e86e5a61, user_id=a6b8c49ae8304c748e6217e3fbfdbf75, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.739484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.752278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1305/2645] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:02 2022] PUT /identity/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61/users/a6b8c49ae8304c748e6217e3fbfdbf75/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:02.756355 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.766291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017780f2-d34e-4bec-b3e6-23c70bde7318 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.824435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.825035 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.825310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.825582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.826211 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.827826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e80b5-ed92-44b8-844f-fc9dfc1bbeb8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.837458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1341/2646] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:35:02.841775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:02.852767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6ce4-296c-45e2-b7cf-15e12dbe83f4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:02.857756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:02.858398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.858680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.858983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] PATH_INFO: `/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61/users/a6b8c49ae8304c748e6217e3fbfdbf75/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.861694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6013de7cea4a9db7e237f7e86e5a61, user_id=a6b8c49ae8304c748e6217e3fbfdbf75, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:02.864338 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:02.874760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1306/2647] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:02 2022] PUT /identity/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61/users/a6b8c49ae8304c748e6217e3fbfdbf75/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:02.885149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:02.885438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:02.885732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:02.914326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] MFA Rules not processed for user `a6b8c49ae8304c748e6217e3fbfdbf75`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:02.915519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:02.977751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:02.989050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1342/2648] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:03.000932 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:03.002652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:03.061617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a59a3-b57f-4e1e-b2ce-b23a6265b75e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:03.065993 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:03.066606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:03.066889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:03.067272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:03.076974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:03.079191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:03.089026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1307/2649] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:03.167120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:03.177056 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a921f1b-58d8-4d5d-be0b-6fca4b4ab915 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:03.180656 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:03.181255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:03.181536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:03.181807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:03.238715 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:03.240109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7d15-d5d4-4bc0-9d2e-049f7ede6885 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:03.248886 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1343/2650] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:13.084729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:13.244814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=31 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:14.592730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:14.667678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1ff6-5a38-485a-9ef1-65471d84a7f1 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:14.673751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:14.674329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:14.674609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:14.674884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] PATH_INFO: `/v3/users/8415b27415d24df1ab6f0e2cdabe070a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:14.676472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8415b27415d24df1ab6f0e2cdabe070a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:14.678381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:14.727918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8415b27415d24df1ab6f0e2cdabe070a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:14.728566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8415b27415d24df1ab6f0e2cdabe070a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:14.736461 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8415b27415d24df1ab6f0e2cdabe070a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:14.748787 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8415b27415d24df1ab6f0e2cdabe070a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:14.749505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8415b27415d24df1ab6f0e2cdabe070a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:14.764695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1308/2651] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:14 2022] DELETE /identity/v3/users/8415b27415d24df1ab6f0e2cdabe070a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:14.769334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:14.769853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:14.769966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:14.790837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:14.792067 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:14.862291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1344/2652] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:14.871674 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:14.873134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:14.929554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425618e-3453-4346-a921-11bc0f6b3999 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:14.933058 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:14.933646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:14.933921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:14.934188 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:14.943030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:14.944456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:14.954800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1309/2653] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:15.227373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:15.229000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:15.241806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43331a-4333-4ae9-8898-9c14690ec106 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:15.246543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:15.247181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:15.247462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:15.247775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] PATH_INFO: `/v3/projects/08340ef11a7149ed9bc3bb84c68bc15c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:15.249039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08340ef11a7149ed9bc3bb84c68bc15c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:15.251144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:15.298661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08340ef11a7149ed9bc3bb84c68bc15c', 'request_id': 'req-cbff3fab-d0ca-4951-a96d-895ff9df678f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.306428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08340ef11a7149ed9bc3bb84c68bc15c', 'request_id': 'req-cbff3fab-d0ca-4951-a96d-895ff9df678f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.313379 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08340ef11a7149ed9bc3bb84c68bc15c', 'request_id': 'req-cbff3fab-d0ca-4951-a96d-895ff9df678f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.324678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] The token cache is being invalidate because project 08340ef11a7149ed9bc3bb84c68bc15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:15.324962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.325588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.327037 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1345/2654] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:15 2022] DELETE /identity/v3/projects/08340ef11a7149ed9bc3bb84c68bc15c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:15.331666 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:15.341170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb848f3-d244-42a1-b28c-d7eda211f394 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:15.358586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:15.399861 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:15.400481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:15.400760 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:15.401030 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] PATH_INFO: `/v3/users/a6b8c49ae8304c748e6217e3fbfdbf75` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:15.402140 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b8c49ae8304c748e6217e3fbfdbf75)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:15.404611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:15.449369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b8c49ae8304c748e6217e3fbfdbf75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.450006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b8c49ae8304c748e6217e3fbfdbf75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.457578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b8c49ae8304c748e6217e3fbfdbf75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.469656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b8c49ae8304c748e6217e3fbfdbf75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.470249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b8849a57-ff7e-4c56-812e-e9b4f574cc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b8c49ae8304c748e6217e3fbfdbf75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.483032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1310/2655] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:15 2022] DELETE /identity/v3/users/a6b8c49ae8304c748e6217e3fbfdbf75 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:15.751337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:15.752844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:15.768575 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff3fab-d0ca-4951-a96d-895ff9df678f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:15.835759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:15.836467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:15.836658 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:15.836929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] PATH_INFO: `/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:15.838112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6013de7cea4a9db7e237f7e86e5a61)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:15.840027 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:15.877014 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:15.894867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6013de7cea4a9db7e237f7e86e5a61', 'request_id': 'req-0884aee7-b13b-4e32-9fb5-1f21583c1378'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.902369 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6013de7cea4a9db7e237f7e86e5a61', 'request_id': 'req-0884aee7-b13b-4e32-9fb5-1f21583c1378'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.910278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6013de7cea4a9db7e237f7e86e5a61', 'request_id': 'req-0884aee7-b13b-4e32-9fb5-1f21583c1378'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.922006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] The token cache is being invalidate because project 7c6013de7cea4a9db7e237f7e86e5a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:15.922291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.922951 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0884aee7-b13b-4e32-9fb5-1f21583c1378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:15.924453 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1346/2656] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:15 2022] DELETE /identity/v3/projects/7c6013de7cea4a9db7e237f7e86e5a61 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:15.931429 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:15.932187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:15.932462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:15.952315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:15.953580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:16.012829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1311/2657] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:16.022081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.022371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.022649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.043759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:16.044973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:16.054143 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1347/2658] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:16.058409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.068686 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d1d82-177a-440b-bcc2-441e705e05ba None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.073248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.073840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.074119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.074390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.075060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:16.075361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.076967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.089234 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1312/2659] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:16.094353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.103666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b75d0-1b69-4aa3-ad24-8db900196708 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.108607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.109948 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.110280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.110495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.111140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.113060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.152130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1348/2660] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:16.157673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.167605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32593b-4dab-4eeb-a181-f83153b70046 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.193766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:16.227800 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.228457 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.228730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.229133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.229807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.231755 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.254396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1313/2661] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:16.258660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.269146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1ca54-df45-49a5-945d-e77e65712394 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.273710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.274305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.274589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.274863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.275511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.277216 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.288265 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1349/2662] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:16.292200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.302865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26a855-befa-4a65-8431-a839d1ffbc00 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.307954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.308543 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.308871 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.309200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.309869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.311446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffccb7f-6b2c-4ff4-8cec-1568fb1be5a5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.324613 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1314/2663] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:16.329804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.340837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc4f87-926f-4bb3-93e9-a3351696c0c1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.347920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.348692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.349043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.349396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] PATH_INFO: `/v3/projects/910ebf90476d406282d9d62d380f5230/users/5cf7a423170745acabbc4daa7e26d905/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.367190 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910ebf90476d406282d9d62d380f5230, user_id=5cf7a423170745acabbc4daa7e26d905, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.369963 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.383061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1350/2664] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:16 2022] PUT /identity/v3/projects/910ebf90476d406282d9d62d380f5230/users/5cf7a423170745acabbc4daa7e26d905/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:16.394431 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956d72-d20a-4519-b536-386b0550794d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.394721 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956d72-d20a-4519-b536-386b0550794d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.395002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956d72-d20a-4519-b536-386b0550794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.425116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-68956d72-d20a-4519-b536-386b0550794d None None] MFA Rules not processed for user `5cf7a423170745acabbc4daa7e26d905`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:16.426343 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-68956d72-d20a-4519-b536-386b0550794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:16.491077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1315/2665] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:16.496520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.560713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a209eae-97cd-4cc2-ab37-0857af952c03 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.566038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.566629 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.566905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.567173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.567841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.569727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.595282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1351/2666] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:16.599597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956d72-d20a-4519-b536-386b0550794d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.611201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956d72-d20a-4519-b536-386b0550794d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.671744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.672621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.672950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.673171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.673801 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.675717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.698128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1316/2667] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:16.702242 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.712784 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed1207-2bfc-4bc0-8c26-22323da980dd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.717438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.718156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.718435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.718707 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.719346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.720969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.731562 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1352/2668] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:16.736357 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.747094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfa61-8d05-47f6-9124-f44c212929d4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.752867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.753516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.753799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.754072 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] PATH_INFO: `/v3/projects/e8f084cc25dd464ea888062cea3bad6e/users/47bed296b21b411b861c9a58336056ff/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.768924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f084cc25dd464ea888062cea3bad6e, user_id=47bed296b21b411b861c9a58336056ff, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.771553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.785566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1317/2669] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:16 2022] PUT /identity/v3/projects/e8f084cc25dd464ea888062cea3bad6e/users/47bed296b21b411b861c9a58336056ff/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:16.789588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.800282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51516a2a-abdf-4d6b-8dda-4b6fcbe8f9f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.811780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:16.861296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.861959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.862173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.862447 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.863085 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.864714 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a31561-da1a-4bdb-98cc-39c612ef53d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.874577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1353/2670] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:16.878882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:16.888161 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:16.895102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a380c-4458-4d1b-953e-da6089720df9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:16.901061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:16.902130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.902656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.903164 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] PATH_INFO: `/v3/projects/e8f084cc25dd464ea888062cea3bad6e/users/47bed296b21b411b861c9a58336056ff/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.906816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f084cc25dd464ea888062cea3bad6e, user_id=47bed296b21b411b861c9a58336056ff, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:16.909742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:16.920105 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1318/2671] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:16 2022] PUT /identity/v3/projects/e8f084cc25dd464ea888062cea3bad6e/users/47bed296b21b411b861c9a58336056ff/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:16.932205 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:16.932491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:16.932767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:16.960564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] MFA Rules not processed for user `47bed296b21b411b861c9a58336056ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:16.961763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:17.029099 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1354/2672] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:17.041565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:17.043406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:17.107098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd2dad-bd24-46ed-bcff-5980aec5e41f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:17.113057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:17.113715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:17.113938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:17.114397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:17.124853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:17.126272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:17.136132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1319/2673] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:17.229284 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:17.239115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9ebf6-4453-4dc7-befc-35b07c72d938 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:17.242675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:17.243272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:17.243556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:17.243844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:17.302603 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:17.304203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:17.313327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1355/2674] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:17.444252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:17.445856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:17.508869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058ad79-5a74-40eb-9765-cd7c26f858e7 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:17.512981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:17.513601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:17.513881 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:17.514151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:17.516528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:17.585809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:17.587243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120ce62-8db1-449d-8960-de093f82b0e8 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:17.595215 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:17.599734 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1320/2675] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:17.636140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:17.662985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5a8e-5db9-4b11-a104-6aba0d24a9b6 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:17.668048 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:17.668601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:17.668944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:17.669357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] PATH_INFO: `/v3/users/5cf7a423170745acabbc4daa7e26d905` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:17.670479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf7a423170745acabbc4daa7e26d905)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:17.672507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:17.722274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf7a423170745acabbc4daa7e26d905'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:17.723009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf7a423170745acabbc4daa7e26d905'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:17.730666 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf7a423170745acabbc4daa7e26d905'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:17.744552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf7a423170745acabbc4daa7e26d905'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:17.745269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf7a423170745acabbc4daa7e26d905'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:17.758289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1356/2676] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:17 2022] DELETE /identity/v3/users/5cf7a423170745acabbc4daa7e26d905 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:35:17.762948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:17.763232 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:17.763522 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:17.784122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:17.785330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:17.826808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:17.850083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1321/2677] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:17.860711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:17.862114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:18.048544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3916c5c-7cb7-4021-aede-3e95ea86b128 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:18.052450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:18.053053 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:18.053326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:18.053593 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:18.063706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:18.065206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:18.077521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1357/2678] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:35:18.356958 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:18.358520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:18.379278 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae78ef-b453-452c-a383-b08a84c960b3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:18.386477 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:18.387790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:18.388506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:18.388790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] PATH_INFO: `/v3/projects/910ebf90476d406282d9d62d380f5230` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:18.390635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=910ebf90476d406282d9d62d380f5230)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:18.393200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:18.440623 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910ebf90476d406282d9d62d380f5230', 'request_id': 'req-efc877fa-a037-4a76-8fa6-025aff7ea113'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.448184 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910ebf90476d406282d9d62d380f5230', 'request_id': 'req-efc877fa-a037-4a76-8fa6-025aff7ea113'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.455969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910ebf90476d406282d9d62d380f5230', 'request_id': 'req-efc877fa-a037-4a76-8fa6-025aff7ea113'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.466452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] The token cache is being invalidate because project 910ebf90476d406282d9d62d380f5230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:18.466743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.467384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.469240 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1322/2679] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:18 2022] DELETE /identity/v3/projects/910ebf90476d406282d9d62d380f5230 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:18.473208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:18.483351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bb08-67c2-4980-8c87-0c81db24c77e service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:18.536892 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:18.544883 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:18.545510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:18.546336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:18.546708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] PATH_INFO: `/v3/users/47bed296b21b411b861c9a58336056ff` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:18.548162 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bed296b21b411b861c9a58336056ff)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:18.550504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:18.613135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bed296b21b411b861c9a58336056ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.613980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bed296b21b411b861c9a58336056ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.622012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bed296b21b411b861c9a58336056ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.635943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bed296b21b411b861c9a58336056ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.636730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-17084e71-114b-481d-8cdd-c6d23c44d18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bed296b21b411b861c9a58336056ff'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:18.653196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1358/2680] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:18 2022] DELETE /identity/v3/users/47bed296b21b411b861c9a58336056ff => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:19.286265 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.287850 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:19.300716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc877fa-a037-4a76-8fa6-025aff7ea113 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.365845 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.366565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.366910 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.367255 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] PATH_INFO: `/v3/projects/e8f084cc25dd464ea888062cea3bad6e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.368483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f084cc25dd464ea888062cea3bad6e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.370451 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.431790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f084cc25dd464ea888062cea3bad6e', 'request_id': 'req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:19.440960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f084cc25dd464ea888062cea3bad6e', 'request_id': 'req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:19.448473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f084cc25dd464ea888062cea3bad6e', 'request_id': 'req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:19.457117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:19.470200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] The token cache is being invalidate because project e8f084cc25dd464ea888062cea3bad6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:19.470549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:19.471405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9f72b297-3ae2-4820-8fcd-c0be9a6ce717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:19.473658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1323/2681] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:19 2022] DELETE /identity/v3/projects/e8f084cc25dd464ea888062cea3bad6e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:19.478783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.479243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.479519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.504783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:19.506184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:19.570461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1359/2682] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:19.579127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.579564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.579943 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.600965 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:19.602314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:19.615300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1324/2683] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:19.616599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.626942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d338c-bdb2-4ce7-ab64-b393ad9a7e7c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.631933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.632713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.633057 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.633391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.634123 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:19.634484 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.636176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.648813 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1360/2684] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:19.653165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.664147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f6ebe-f19b-4eb5-a64e-3a02af467294 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.668540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.669152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.669497 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.670199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.670409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.673237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.705840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1325/2685] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:19 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:19.710101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.722611 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92608ebd-305d-401a-b1aa-3bc2300019f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.784937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.785608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.785851 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.786077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.786720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.788727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.799526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:19.818680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1361/2686] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:19 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:35:19.822720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.833752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7382a0-e87f-4f31-854d-a9b905401a9c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.838520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.839136 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.839418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.839727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.840372 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.841956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.852911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1326/2687] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:19.856834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.870620 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf4bff-1b16-4cd2-9e14-5e7e2bcc8876 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.876962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.877716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.878250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.878454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.879313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.881142 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a520ff-9bc0-45fb-badb-8396a513a5f0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.894942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1362/2688] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:19.897282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:19.907875 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915fe4-f4e6-4450-9898-112fc518b758 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:19.912964 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:19.913696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.914076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.914472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] PATH_INFO: `/v3/projects/5618983fcf574c4ab9c6b2927b19fad5/users/b44a1af6e2c84543934f7bc3660d8445/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.930421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5618983fcf574c4ab9c6b2927b19fad5, user_id=b44a1af6e2c84543934f7bc3660d8445, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:19.933219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:19.962485 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1327/2689] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:19 2022] PUT /identity/v3/projects/5618983fcf574c4ab9c6b2927b19fad5/users/b44a1af6e2c84543934f7bc3660d8445/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:19.964385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:19.964768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:19.965139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:19.995455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] MFA Rules not processed for user `b44a1af6e2c84543934f7bc3660d8445`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:19.997032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:20.081660 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1363/2690] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:20.089365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:20.091122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:20.158907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909247dd-3ffb-4753-bc19-8536338fb295 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:20.162536 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:20.163138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:20.163416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:20.163713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:20.172834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:20.174251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5a16c-c8cb-4e7d-8336-24a85516e2aa service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:20.183096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1328/2691] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:35:28.628492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:28.699034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaeaa9d-6518-48e4-8366-c5169356cf46 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:28.704608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:28.705235 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:28.705516 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:28.705790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] PATH_INFO: `/v3/users/b44a1af6e2c84543934f7bc3660d8445` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:28.706929 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44a1af6e2c84543934f7bc3660d8445)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:28.708890 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:28.756469 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44a1af6e2c84543934f7bc3660d8445'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:28.757117 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44a1af6e2c84543934f7bc3660d8445'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:28.764127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44a1af6e2c84543934f7bc3660d8445'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:28.776450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44a1af6e2c84543934f7bc3660d8445'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:28.777087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44a1af6e2c84543934f7bc3660d8445'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:28.779017 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:28.791241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1364/2692] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:28 2022] DELETE /identity/v3/users/b44a1af6e2c84543934f7bc3660d8445 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:35:28.795905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:28.796396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:28.796496 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:28.815806 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:28.817000 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:28.876022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1329/2693] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:28.885648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:28.887046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:28.947445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5888c-b469-4be4-9a11-e2c1b8cf9441 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:28.951134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:28.951775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:28.952055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:28.952390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:28.962461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:28.964002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56f17e-b194-46df-a00a-86918f870832 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:28.973456 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1365/2694] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:29.244824 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.246326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:29.258772 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170232a8-e366-4d33-a61a-7e8cd557b80f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.264167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.264167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.264408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.264665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] PATH_INFO: `/v3/projects/5618983fcf574c4ab9c6b2927b19fad5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.265976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5618983fcf574c4ab9c6b2927b19fad5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.267889 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.303210 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:29.317884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5618983fcf574c4ab9c6b2927b19fad5', 'request_id': 'req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:29.325130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5618983fcf574c4ab9c6b2927b19fad5', 'request_id': 'req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:29.331561 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5618983fcf574c4ab9c6b2927b19fad5', 'request_id': 'req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:29.341596 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] The token cache is being invalidate because project 5618983fcf574c4ab9c6b2927b19fad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:29.341878 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:29.342495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2bb4a122-2ec9-4dc9-b08d-b5a3df4e0ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:29.344217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1330/2695] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:29 2022] DELETE /identity/v3/projects/5618983fcf574c4ab9c6b2927b19fad5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:29.349055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.349360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.349636 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.369317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:29.370511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:29.427907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1366/2696] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:29.436617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.436898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.437169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.458106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:29.459313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:29.467305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1331/2697] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:29.471637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.481161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791834a-3316-4bae-a493-6d942bbe26e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.486612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.487202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.487481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.487780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.488419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:29.488718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.490286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.502724 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1367/2698] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:29.506989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.517143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea32321-7616-4637-b834-3fa3374c4c06 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.521852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.522440 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.522715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.522980 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.523671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.525854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.554954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1332/2699] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:29 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:35:29.559032 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.569814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9798b2-e49c-47bb-80ec-c98fde86cf64 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.626792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.627415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.627733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.628006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.628642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.630559 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.652536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1368/2700] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:29 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:29.657138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.667334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd31498-f913-4e3e-865d-966bfd570272 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.671989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.672592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.672874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.673146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.673784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.675361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.685173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1333/2701] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:29.689522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.699350 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccd3a0-3840-4aee-9b9c-2c9bdeab1aea admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.704926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.705515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.705797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.706072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.706840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.708458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd959bcc-4006-4fa4-933b-6d545fd0ee2f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.713544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:29.720271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1369/2702] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:35:29.724116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.734078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc49b-bbcd-42a8-bf53-ccb65522130b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.738947 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.739619 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.739904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.740178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] PATH_INFO: `/v3/projects/edb94a4b7fb248a8be7a2a18b6213de9/users/e3989993368b4e7982386f4622e986b0/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.755015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb94a4b7fb248a8be7a2a18b6213de9, user_id=e3989993368b4e7982386f4622e986b0, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.757685 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.770931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1334/2703] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:29 2022] PUT /identity/v3/projects/edb94a4b7fb248a8be7a2a18b6213de9/users/e3989993368b4e7982386f4622e986b0/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:29.781584 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.782061 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.782372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.809563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] MFA Rules not processed for user `e3989993368b4e7982386f4622e986b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:29.810757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:29.879033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1370/2704] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:29.889742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:29.891754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:29.954409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5730bc1-5f02-4aa0-ba0f-64816496a885 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:29.958519 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:29.959167 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:29.959469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:29.959747 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:29.969673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:29.971103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef89909-034d-4e1b-9a6a-c2af93c6c247 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:29.980147 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1335/2705] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:35:33.015648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.077292 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec64a0-cc48-49fb-8d86-3120aa661d60 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:33.081916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:33.082530 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.082808 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.083078 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] PATH_INFO: `/v3/users/e3989993368b4e7982386f4622e986b0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.084203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3989993368b4e7982386f4622e986b0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:33.086116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:33.131079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3989993368b4e7982386f4622e986b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.131765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3989993368b4e7982386f4622e986b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.138526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3989993368b4e7982386f4622e986b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.150732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3989993368b4e7982386f4622e986b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.151357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3989993368b4e7982386f4622e986b0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.164139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1371/2706] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:33 2022] DELETE /identity/v3/users/e3989993368b4e7982386f4622e986b0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:33.168749 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.169211 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.169326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.188406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:33.190273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:33.239201 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:33.250426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1336/2707] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:33.260378 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.261796 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:33.272800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:33.320574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570e4a1-fadb-4ba9-b938-a8a3f93ed028 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:33.324147 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:33.324746 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.325019 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.325289 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.334785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:33.336232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fa2b4-308d-48b0-b50f-cc63b2d85d48 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:33.346324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1372/2708] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:33.617529 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.618945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:33.640325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ca2b2-049b-4505-b698-52af6d67f969 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:33.646816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:33.647463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.647784 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.648060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] PATH_INFO: `/v3/projects/edb94a4b7fb248a8be7a2a18b6213de9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.649195 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb94a4b7fb248a8be7a2a18b6213de9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:33.651123 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:33.696669 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb94a4b7fb248a8be7a2a18b6213de9', 'request_id': 'req-232df1cb-4f36-43bd-9168-93eb99bd00cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.703727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb94a4b7fb248a8be7a2a18b6213de9', 'request_id': 'req-232df1cb-4f36-43bd-9168-93eb99bd00cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.710904 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb94a4b7fb248a8be7a2a18b6213de9', 'request_id': 'req-232df1cb-4f36-43bd-9168-93eb99bd00cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.722368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] The token cache is being invalidate because project edb94a4b7fb248a8be7a2a18b6213de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:33.722705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.723345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-232df1cb-4f36-43bd-9168-93eb99bd00cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:33.724867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1337/2709] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:33 2022] DELETE /identity/v3/projects/edb94a4b7fb248a8be7a2a18b6213de9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:33.730111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.730398 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.730702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.750750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:33.752066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:33.810828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1373/2710] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:33.818847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.819153 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.819418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.839652 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:33.840858 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:33.849398 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1338/2711] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:33.853706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.863933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b356-d520-4a7c-876c-75ce6d0e9b40 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:33.868448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:33.869036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.869309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.869576 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.870214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:33.870508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:33.872111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:33.884654 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1374/2712] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:33.888839 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.898238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af474b-7b0e-4be7-b81d-37f3446190bb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:33.902906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:33.903807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:33.904093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:33.904365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:33.904997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:33.906868 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:33.935266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1339/2713] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:33 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:35:33.940004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:33.949754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e69b-3344-41bb-b175-4f895a6005f3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:34.005899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:34.006493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.006769 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.007038 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.007680 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:34.009676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:34.030794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1375/2714] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:33 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:34.035494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:34.045791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fabf-e6bd-43b2-8264-522116b22357 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:34.051170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:34.051804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.052084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.052376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.053021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:34.054586 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:34.064377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1340/2715] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:34.068127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:34.077773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725809-0062-41de-928c-aef9ee8967c9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:34.082546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:34.083125 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.083401 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.083793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.084421 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:34.086243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675f4ad-f3a2-470e-92d5-66a4211563c8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:34.097039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1376/2716] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:34.100952 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:34.111314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4e137-b2dd-4574-ac66-cdf1dd21ebae admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:34.116009 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:34.116657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.116935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.117208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] PATH_INFO: `/v3/projects/8f04ce610aba42359e397b43073c5384/users/b207cacaf82c4e9ca3a911436845ab64/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.131617 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f04ce610aba42359e397b43073c5384, user_id=b207cacaf82c4e9ca3a911436845ab64, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:34.134226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:34.146810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1341/2717] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:34 2022] PUT /identity/v3/projects/8f04ce610aba42359e397b43073c5384/users/b207cacaf82c4e9ca3a911436845ab64/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:34.159232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.159517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.159818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.187712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] MFA Rules not processed for user `b207cacaf82c4e9ca3a911436845ab64`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:34.189168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:34.194154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:34.254786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1377/2718] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:34.266310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:34.268254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:34.325592 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483d95-2b70-446c-9daa-79fd30131c40 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:34.329865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:34.330499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:34.330803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:34.331073 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:34.339383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:34.340834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:34.350238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1342/2719] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:35.912505 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:35.913908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:35.973154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248e29a-51df-4197-8692-9eb10de49980 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:35.976831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:35.977442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:35.977834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:35.978114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:35.980313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:36.054878 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:36.056259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:36.070426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1378/2720] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:37.263189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:37.264628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:37.277086 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40352675-1727-44f5-adf7-e86a29c8c0d4 service glance] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:37.280962 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:37.281562 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:37.281842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:37.282658 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:37.284752 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:37.299120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:37.300568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:37.311435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1343/2721] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:42.838203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:42.846493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6b1d-34bd-41d3-aefd-0c4ab28feafe service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:42.850506 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:42.851112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:42.851383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:42.851755 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:42.853496 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:42.903165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:42.921622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:42.923377 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:42.934740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1379/2722] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:46.271103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:46.272671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:46.287091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddc784-896f-4e45-8832-a4e7dd8bc012 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:46.291732 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:46.292642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.292913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.293179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] PATH_INFO: `/v3/users/85a24cb16a864a03920ccb359c13ec79` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.295373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:46.311415 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a24cb16a864a03920ccb359c13ec79)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:46.314680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:46.363817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a24cb16a864a03920ccb359c13ec79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.364484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a24cb16a864a03920ccb359c13ec79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.371702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a24cb16a864a03920ccb359c13ec79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.384449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a24cb16a864a03920ccb359c13ec79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.385150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5963c497-e59e-4581-8334-4b5b8299ce57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a24cb16a864a03920ccb359c13ec79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.398808 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1344/2723] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:35:46 2022] DELETE /identity/v3/users/85a24cb16a864a03920ccb359c13ec79 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:46.590773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:46.601070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91dde-3243-4b33-acaf-2ecfc207cf43 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:46.660324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:46.660965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.661243 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.661514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] PATH_INFO: `/v3/projects/ec1c332dc5c44eee9c7c81787a435533` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.672282 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1c332dc5c44eee9c7c81787a435533)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:46.674513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:46.719662 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1c332dc5c44eee9c7c81787a435533', 'request_id': 'req-e56703a6-5a8e-42fe-b925-a1d871e3ea78'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.727197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1c332dc5c44eee9c7c81787a435533', 'request_id': 'req-e56703a6-5a8e-42fe-b925-a1d871e3ea78'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.732855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1c332dc5c44eee9c7c81787a435533', 'request_id': 'req-e56703a6-5a8e-42fe-b925-a1d871e3ea78'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.746818 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] The token cache is being invalidate because project ec1c332dc5c44eee9c7c81787a435533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:35:46.747103 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.747798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e56703a6-5a8e-42fe-b925-a1d871e3ea78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:35:46.749278 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1380/2724] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:35:46 2022] DELETE /identity/v3/projects/ec1c332dc5c44eee9c7c81787a435533 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:46.754983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.755266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.755554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.774770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:46.776020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:46.835466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1345/2725] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:46.843326 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.843648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.843924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.867298 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:46.868483 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:46.876903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1381/2726] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:46.881202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:46.890938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225984aa-3b67-4803-8dbe-9c6b228d1443 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:46.895514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:46.896130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.896406 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.896673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.897318 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:35:46.897612 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:46.899189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:46.911178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1346/2727] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:35:46.915462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:46.925385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a27b0-0ee7-48e3-a26d-e9372044a75a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:46.929953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:46.930526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:46.930800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:46.931069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:46.931722 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:46.933623 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:46.960384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1382/2728] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:35:46 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:46.964486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:46.974705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae57fa6-0605-4d8a-9cb6-4771328b0bbb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:47.026043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:47.035309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:47.035956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.036425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.036766 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.037585 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:47.039584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:47.060778 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1347/2729] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:35:46 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:35:47.065954 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:47.076356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930705af-5601-42fe-9422-a178c4a94f8e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:47.081832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:47.082607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.083015 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.083387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.084160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:47.085917 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:47.096773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1383/2730] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:35:47.100873 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:47.111580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0ea15-7a60-4e2a-a236-98cf0f4c7628 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:47.116884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:47.117548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.117777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.118045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.118680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:47.120282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20934d49-c246-4c8a-bb8d-3b63e1bc3a30 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:47.131603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1348/2731] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:35:47.136031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:47.146587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282682-a7be-4c06-9a82-b78f6217979e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:47.151918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:47.152560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.152835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.153106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] PATH_INFO: `/v3/projects/7cca63adb761474d82636997dade783a/users/70dcf397dc04470b9756119a9fe8f6a6/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.169268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cca63adb761474d82636997dade783a, user_id=70dcf397dc04470b9756119a9fe8f6a6, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:47.171900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:47.180596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:35:47.186251 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1384/2732] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:35:47 2022] PUT /identity/v3/projects/7cca63adb761474d82636997dade783a/users/70dcf397dc04470b9756119a9fe8f6a6/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:35:47.192660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2614b5-5917-4ede-a765-a8b8430031f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.193170 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2614b5-5917-4ede-a765-a8b8430031f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.193697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2614b5-5917-4ede-a765-a8b8430031f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.216166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cb2614b5-5917-4ede-a765-a8b8430031f7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:47.218326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb2614b5-5917-4ede-a765-a8b8430031f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:47.286434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1349/2733] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:35:47.296600 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:47.299113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:47.357930 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad848fda-08b0-49e1-a740-2648a910c390 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:47.362137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:47.362733 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:47.363007 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:47.363274 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:47.372264 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:47.373616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:47.384762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1385/2734] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:51.338252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aceae-6fb6-4b93-bda3-8330c6f5f140 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:51.338790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aceae-6fb6-4b93-bda3-8330c6f5f140 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:51.338790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aceae-6fb6-4b93-bda3-8330c6f5f140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:51.368150 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6b4aceae-6fb6-4b93-bda3-8330c6f5f140 None None] MFA Rules not processed for user `70dcf397dc04470b9756119a9fe8f6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:51.369426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6b4aceae-6fb6-4b93-bda3-8330c6f5f140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:51.442992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1350/2735] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:51.457495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:51.458907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:51.541525 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5f0bd-df9b-4737-9eaf-0046f2088480 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:51.547224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:51.547928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:51.548218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:51.548474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:51.557440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:51.559098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:51.569450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1386/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:53.166285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:53.166840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:53.166840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:53.186107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] MFA Rules not processed for user `70dcf397dc04470b9756119a9fe8f6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:35:53.187294 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:53.194771 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1351/2737] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:35:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:35:53.843119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:53.846653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:53.931834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89d8d-b602-4067-aa88-0e700edb06b6 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:53.936521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:53.937255 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:53.937638 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:53.938034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:53.941651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:53.957898 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:53.960070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:53.971624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1387/2738] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:35:56.039793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:35:56.051169 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22673360-438f-401e-906d-75e8a4b24d07 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:35:56.055254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:35:56.055945 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:35:56.056249 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:35:56.056516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:35:56.058324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:35:56.070434 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:35:56.072095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12334917-ea86-4d39-8d59-23bd5e3a93ba service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:35:56.081896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1352/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:35:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:36:03.966538 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:06.077935 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:07.749901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:07.751324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:07.765479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091fa2a-63ff-4621-8696-eda92428c9da service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:07.769977 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:07.770615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:07.770887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:07.771156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] PATH_INFO: `/v3/users/b207cacaf82c4e9ca3a911436845ab64` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:07.772296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b207cacaf82c4e9ca3a911436845ab64)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:07.774182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:07.820855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b207cacaf82c4e9ca3a911436845ab64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:07.821492 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b207cacaf82c4e9ca3a911436845ab64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:07.828250 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b207cacaf82c4e9ca3a911436845ab64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:07.839900 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b207cacaf82c4e9ca3a911436845ab64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:07.840519 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b207cacaf82c4e9ca3a911436845ab64'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:07.852564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1388/2740] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:36:07 2022] DELETE /identity/v3/users/b207cacaf82c4e9ca3a911436845ab64 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:36:07.857219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:07.857730 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:07.857840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:07.876803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:07.877996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:07.943326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1353/2741] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:07.951213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:07.952604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.012159 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4104-c257-423e-8908-4cf1374178bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.015811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.016429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.016701 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.016972 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.026992 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.028474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e4b5a-7982-4b44-ae9e-e2cbc7262f36 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.039695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1389/2742] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:36:08.315458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.317401 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.330409 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f93520-3442-4916-b180-cca549750ba7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.335187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.335972 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.336322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.336606 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] PATH_INFO: `/v3/projects/8f04ce610aba42359e397b43073c5384` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.337770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f04ce610aba42359e397b43073c5384)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.339780 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.387567 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f04ce610aba42359e397b43073c5384', 'request_id': 'req-a8ecf522-29e5-485e-bedc-90305040f766'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:08.395296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f04ce610aba42359e397b43073c5384', 'request_id': 'req-a8ecf522-29e5-485e-bedc-90305040f766'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:08.401997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f04ce610aba42359e397b43073c5384', 'request_id': 'req-a8ecf522-29e5-485e-bedc-90305040f766'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:08.412673 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] The token cache is being invalidate because project 8f04ce610aba42359e397b43073c5384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:36:08.412960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:08.413603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a8ecf522-29e5-485e-bedc-90305040f766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:36:08.415103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1354/2743] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:36:08 2022] DELETE /identity/v3/projects/8f04ce610aba42359e397b43073c5384 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:36:08.420182 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.420468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.420740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.440688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:08.442083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.500054 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1390/2744] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:08.508102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.508389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.508665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.528698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:08.530158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.539017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1355/2745] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:08.543189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.553059 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd3a1a-e2cc-400b-9b51-85b2a57e1ade None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.557678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.558272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.558554 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.558831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.559479 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:36:08.559825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.561409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.572857 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1391/2746] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:36:08.577103 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.588817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75cee1-cccb-4260-b2f7-5f901c345235 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.593805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.594628 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.594882 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.595151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.595830 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.598670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.626346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1356/2747] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:36:08 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:36:08.631021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.636258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:08.641532 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e2995-b24b-492b-aaa9-a3e6192f07eb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.704696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.705301 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.705586 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.705859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.706501 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.709127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.730780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1392/2748] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:36:08 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:36:08.734918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.745080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437745a-f945-4292-bd0c-c069c6c8a2ef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.749642 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.750277 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.750499 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.750767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.751397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.753008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.762896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1357/2749] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:36:08.766738 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.777481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11c57-6cba-404a-9d99-e61448ad6e49 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.782394 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.782982 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.783256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.783551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.784194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.786466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a9e3b-b836-4f5b-b3e2-264dbf3196dc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.797202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1393/2750] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:36:08.801549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.811463 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e67683-9304-4f3e-ad28-6cfbd295c498 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:08.816116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:08.817604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.817911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.818185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] PATH_INFO: `/v3/projects/91a4df4888bc427ab3f8195e68604c96/users/4654c5c4e09e4c25ab58aca20cfcc7e2/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.833119 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a4df4888bc427ab3f8195e68604c96, user_id=4654c5c4e09e4c25ab58aca20cfcc7e2, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:08.835764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:08.848703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1358/2751] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:36:08 2022] PUT /identity/v3/projects/91a4df4888bc427ab3f8195e68604c96/users/4654c5c4e09e4c25ab58aca20cfcc7e2/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:36:08.853859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d1a-8845-40ba-ac49-e8b80f262eba None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:08.854192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d1a-8845-40ba-ac49-e8b80f262eba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:08.854468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c5d1a-8845-40ba-ac49-e8b80f262eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:08.880157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-007c5d1a-8845-40ba-ac49-e8b80f262eba None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:08.881354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-007c5d1a-8845-40ba-ac49-e8b80f262eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.938985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1394/2752] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:08.948607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:08.950298 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:08.971870 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:09.012021 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753092-2ef9-4533-a3c3-983db4a512bf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:09.015521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:09.016143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:09.016422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:09.016695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:09.025918 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:09.027334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:09.037566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1359/2753] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:36:13.586410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:13.586968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:13.586968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:13.618842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] MFA Rules not processed for user `4654c5c4e09e4c25ab58aca20cfcc7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:13.620110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:13.690687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1395/2754] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:13.695851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:13.705810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57384b-4f90-4b70-ab46-fa4bb82a9329 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:13.710741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:13.711345 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:13.711659 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:13.711936 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:13.712578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:13.714482 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:13.743046 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1360/2755] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:36:13 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:36:13.747276 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:13.758967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b22e5-43eb-47e0-82c3-e6fe05f6301f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:13.767066 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:13.821031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:13.821645 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:13.821928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:13.822203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:13.823497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:13.825441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:13.847770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1396/2756] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:36:13 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:36:13.852411 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:13.868649 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8614e-26fd-4383-a58b-a4ba57281be0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:13.873464 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:13.874069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:13.874344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:13.874611 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:13.875283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:13.877014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:13.892007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1361/2757] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:36:13.892959 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:13.904296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61976-5797-4ef7-92b3-133288c5cde6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:13.909352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:13.910012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:13.910993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:13.911324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] PATH_INFO: `/v3/projects/d54cf63558bf4eb08ae32bc6c1817587/users/74bf484d6d744eff806bbdcaf7a55d80/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:13.927179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54cf63558bf4eb08ae32bc6c1817587, user_id=74bf484d6d744eff806bbdcaf7a55d80, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:13.929777 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:13.943313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1397/2758] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:36:13 2022] PUT /identity/v3/projects/d54cf63558bf4eb08ae32bc6c1817587/users/74bf484d6d744eff806bbdcaf7a55d80/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:36:13.947321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:13.958246 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73204c-120a-4680-8881-4934cf968994 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:14.018442 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:14.019056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:14.019347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:14.020230 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:14.020884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:14.022466 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46974538-6779-4b40-85ec-85cf49efd312 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:14.032886 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1362/2759] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:36:14.038458 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:14.050445 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613960-9d7f-4391-9a78-ff1e08516cc9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:14.055435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:14.056110 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:14.056392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:14.056667 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] PATH_INFO: `/v3/projects/d54cf63558bf4eb08ae32bc6c1817587/users/74bf484d6d744eff806bbdcaf7a55d80/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:14.058565 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54cf63558bf4eb08ae32bc6c1817587, user_id=74bf484d6d744eff806bbdcaf7a55d80, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:14.061201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:14.072097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1398/2760] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:36:14 2022] PUT /identity/v3/projects/d54cf63558bf4eb08ae32bc6c1817587/users/74bf484d6d744eff806bbdcaf7a55d80/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:36:18.360989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016f82-4ca6-4d71-a13b-671bf22c9c90 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:18.361455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016f82-4ca6-4d71-a13b-671bf22c9c90 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:18.361635 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016f82-4ca6-4d71-a13b-671bf22c9c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:18.399028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-16016f82-4ca6-4d71-a13b-671bf22c9c90 None None] MFA Rules not processed for user `74bf484d6d744eff806bbdcaf7a55d80`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:18.400309 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-16016f82-4ca6-4d71-a13b-671bf22c9c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:18.487818 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1363/2761] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:18.499682 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:18.501468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:18.560329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563440e3-c621-4929-b28e-bf1a2c0fc99d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:18.564002 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8985883a056c431b885bb07303d3f8af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:18.564604 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:18.564882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:18.565269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:18.626451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:18.627842 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:18.642069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1399/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:36:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:36:18.796494 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d61e3-0c66-4b8e-999c-ef54f1be4621 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:18.796987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d61e3-0c66-4b8e-999c-ef54f1be4621 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:18.797263 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d61e3-0c66-4b8e-999c-ef54f1be4621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:18.828587 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:18.868224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-257d61e3-0c66-4b8e-999c-ef54f1be4621 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:18.869473 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-257d61e3-0c66-4b8e-999c-ef54f1be4621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:18.940197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1364/2763] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:36:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:18.992009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:18.993428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:19.034773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:19.117786 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9d49-3b7d-4c49-95e7-b3373c182397 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:19.125119 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:19.125743 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:19.126022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:19.126295 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:19.135965 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:19.137558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a128350-77bb-408e-bc22-27cc034ac38b service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:19.146642 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1400/2764] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:36:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:36:28.003660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:28.004181 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:28.004282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:28.024028 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] MFA Rules not processed for user `4654c5c4e09e4c25ab58aca20cfcc7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:28.025238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:28.032310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1365/2765] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:36:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:29.142810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=33 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:30.128196 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:30.128710 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:30.129068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:30.154376 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:36:30.155616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:36:30.164006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1401/2766] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 21:36:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:36:30.177069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:36:30.189389 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09952a64-b837-4b77-856e-10366eb0acc1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:36:30.195059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:36:30.195680 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:36:30.195960 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:36:30.196226 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:36:30.204424 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:36:30.206004 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:36:30.214402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1366/2767] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:36:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:36:40.148322 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:36:40.210867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=44 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:37:01.193407 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:37:01.246423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897971d-816f-4834-bb4f-6a5ec3c081f9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:37:01.249968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:37:01.250560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:01.250840 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:01.251106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:01.260653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:37:01.262126 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6bdad-4868-4e26-929a-f495dc9b396a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:37:01.271865 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1402/2768] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:37:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:37:02.844576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:37:02.846102 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:02.930383 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68d12-cbf5-45f9-8c0d-e5b88c86578d service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:37:02.937145 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:37:02.937814 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:02.938100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:02.938374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:02.940288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:02.953847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:37:02.955506 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:37:02.967667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1367/2769] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:37:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:37:11.267571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=26 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:37:12.963245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=34 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:37:36.833993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:36.834514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:36.834618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:36.853989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] MFA Rules not processed for user `70dcf397dc04470b9756119a9fe8f6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:37:36.855189 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:36.909535 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1403/2770] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:37:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:37:37.221458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:37:37.230716 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729e34-3494-46c7-980e-6de5e0a41b6b service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:37:37.234565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:37:37.235165 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:37.235446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:37.235745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:37.237521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:37.247428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:37:37.248898 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:37:37.256890 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1368/2771] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:37:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:37:40.334031 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:37:40.335741 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:40.353468 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab4dad-6b04-4c36-857d-442bfa5d8e30 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:37:40.359712 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:37:40.360329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:40.360634 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:40.360905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:40.362632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:40.383431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:37:40.385194 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6b29-ad26-4625-843b-6f780eafe202 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:37:40.396466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1404/2772] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:37:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:37:45.145143 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:37:45.159236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390d09-fea7-46bf-a753-d7bf59aa3cfc service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:37:45.164981 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:37:45.165576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:37:45.165849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:37:45.166149 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:37:45.167874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:37:45.181737 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:37:45.183742 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:37:45.195634 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1369/2773] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:37:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:37:50.391766 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=31 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:37:55.191484 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:38:01.442092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053de1-fce0-49b0-986e-bfa8c3fc8f79 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:38:01.442577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053de1-fce0-49b0-986e-bfa8c3fc8f79 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:38:01.442695 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053de1-fce0-49b0-986e-bfa8c3fc8f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:38:01.462592 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cf053de1-fce0-49b0-986e-bfa8c3fc8f79 None None] MFA Rules not processed for user `691a25c97f4748119366c35dd4688765`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:38:01.463809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cf053de1-fce0-49b0-986e-bfa8c3fc8f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:38:01.517631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1405/2774] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:38:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:38:11.510373 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:38:32.368615 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:38:32.370114 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:38:32.437692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f1902-89bf-4aba-b359-da7e1cad4084 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:38:32.442781 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:38:32.443387 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:38:32.443697 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:38:32.443977 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:38:32.445728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:38:32.457208 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:38:32.458765 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf13c80-61d7-4253-a94c-f2f5056d2533 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:38:32.469272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1370/2775] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:38:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:38:42.464353 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=32 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:39:17.147328 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:17.147964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:17.147964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:17.167241 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] MFA Rules not processed for user `70dcf397dc04470b9756119a9fe8f6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:39:17.168475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:39:17.175952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1406/2776] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:39:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:39:27.161351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:39:32.519245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:32.520243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:32.520243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:32.540588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] MFA Rules not processed for user `aae0056168e0416c82c5cdba519b9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:39:32.541896 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:39:32.549715 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1371/2777] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 21:39:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:39:32.593702 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:32.606877 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360178b0-dbb8-4ac6-8bb0-a917322cd482 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:32.612096 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103b5c20d7354245a2c57adb0363ffce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:32.612713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:32.612985 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:32.613384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:32.622587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:32.624225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:32.633363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1407/2778] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 21:39:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 21:39:39.636698 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:39.651520 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc97375-490a-4896-a983-fe1cf8c254e6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:39.657012 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:39.657503 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:39.657779 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:39.658132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:39.659829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:39:39.675252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:39.678331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:39.700189 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1372/2779] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:39:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:39:39.705973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:39.770010 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7128-0ad4-49e6-aca5-5a7ac9d80fa9 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:39.774426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:39.775043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:39.775317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:39.775663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] PATH_INFO: `/v3/users/4654c5c4e09e4c25ab58aca20cfcc7e2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:39.779517 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4654c5c4e09e4c25ab58aca20cfcc7e2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:39.781450 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:39.826711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4654c5c4e09e4c25ab58aca20cfcc7e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:39.827343 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4654c5c4e09e4c25ab58aca20cfcc7e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:39.834060 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4654c5c4e09e4c25ab58aca20cfcc7e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:39.845423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4654c5c4e09e4c25ab58aca20cfcc7e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:39.846052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4654c5c4e09e4c25ab58aca20cfcc7e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:39.858955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1408/2780] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:39:39 2022] DELETE /identity/v3/users/4654c5c4e09e4c25ab58aca20cfcc7e2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:39:40.028154 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:40.040223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb88-4559-4136-9fbd-82564666d80f service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:40.103651 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:40.104286 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:40.104560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:40.104833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] PATH_INFO: `/v3/projects/91a4df4888bc427ab3f8195e68604c96` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:40.105973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a4df4888bc427ab3f8195e68604c96)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:40.107908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:40.154750 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a4df4888bc427ab3f8195e68604c96', 'request_id': 'req-be6406c6-85c2-4e61-a22b-10606e775a86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.162761 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a4df4888bc427ab3f8195e68604c96', 'request_id': 'req-be6406c6-85c2-4e61-a22b-10606e775a86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.169804 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a4df4888bc427ab3f8195e68604c96', 'request_id': 'req-be6406c6-85c2-4e61-a22b-10606e775a86'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.180113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] The token cache is being invalidate because project 91a4df4888bc427ab3f8195e68604c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:39:40.180403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.181059 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.182526 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1373/2781] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:39:40 2022] DELETE /identity/v3/projects/91a4df4888bc427ab3f8195e68604c96 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:39:40.186621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:40.188079 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:39:40.202012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7bb3-6e50-40dc-b695-b0c05863a23e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:40.259602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:40.260222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:40.260495 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:40.260764 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] PATH_INFO: `/v3/users/74bf484d6d744eff806bbdcaf7a55d80` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:40.261882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bf484d6d744eff806bbdcaf7a55d80)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:40.263807 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:40.268995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:39:40.310127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bf484d6d744eff806bbdcaf7a55d80'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.310793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bf484d6d744eff806bbdcaf7a55d80'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.317750 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bf484d6d744eff806bbdcaf7a55d80'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.329311 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bf484d6d744eff806bbdcaf7a55d80'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.329943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bf484d6d744eff806bbdcaf7a55d80'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.342631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1409/2782] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:39:40 2022] DELETE /identity/v3/users/74bf484d6d744eff806bbdcaf7a55d80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:39:40.516865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:39:40.518368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:39:40.532591 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6406c6-85c2-4e61-a22b-10606e775a86 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:39:40.592727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:39:40.593582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:39:40.593940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:39:40.594284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] PATH_INFO: `/v3/projects/d54cf63558bf4eb08ae32bc6c1817587` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:39:40.595435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54cf63558bf4eb08ae32bc6c1817587)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:39:40.597376 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:39:40.609903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:39:40.648003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54cf63558bf4eb08ae32bc6c1817587', 'request_id': 'req-90d20f51-b8f3-4871-baa1-45a5878b8c8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.655257 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54cf63558bf4eb08ae32bc6c1817587', 'request_id': 'req-90d20f51-b8f3-4871-baa1-45a5878b8c8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.661903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54cf63558bf4eb08ae32bc6c1817587', 'request_id': 'req-90d20f51-b8f3-4871-baa1-45a5878b8c8d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.672435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] The token cache is being invalidate because project d54cf63558bf4eb08ae32bc6c1817587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:39:40.672719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.673355 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:39:40.674836 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1374/2783] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:39:40 2022] DELETE /identity/v3/projects/d54cf63558bf4eb08ae32bc6c1817587 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:39:50.339183 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:39:50.669361 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:40:36.537905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:36.539366 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:36.606336 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6f51-9cc4-4b68-9a8a-d844a03aab33 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:36.656246 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:36.656852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:36.657129 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:36.657396 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:36.659114 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:36.716841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:36.718258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b428b7b-db7e-4969-bfc1-0e33c5b8c889 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:36.733950 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1410/2784] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:40:37.662773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:37.664191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:37.677052 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20f51-b8f3-4871-baa1-45a5878b8c8d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:37.680569 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:37.681782 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:37.682100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:37.682362 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:37.684334 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:37.757139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:37.758563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:37.769707 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1375/2785] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:40:46.723081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=68 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:40:47.763663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:47.764272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:47.764272 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:47.764820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=34 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:40:47.783467 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] MFA Rules not processed for user `691a25c97f4748119366c35dd4688765`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:40:47.784730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:47.791581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1411/2786] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:40:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:40:47.832097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:47.833495 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:47.844609 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c383eb-bcb8-4aa2-becf-cf5cc340c51a service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:47.848252 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:47.848842 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:47.849122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:47.849397 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:47.851066 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:47.864956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:47.866370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:47.878625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1376/2787] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:40:48.184135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:48.185543 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:48.241094 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370eb3d-af87-4361-9100-e2672dbcddbb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:48.244635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:48.245566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:48.245885 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:48.246153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:48.247952 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:48.257510 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:48.258923 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758be035-5b46-481a-85dd-89f80acf632d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:48.266866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1412/2788] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:40:51.570280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:51.571788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:51.585899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00f1e-ed3e-41c2-befc-b5f7335e2644 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:51.591692 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:51.592289 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:51.592559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:51.592854 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:51.594505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:51.652621 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:51.654202 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:51.663423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1377/2789] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:40:57.350193 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:57.350708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:57.350826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:57.371463 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] MFA Rules not processed for user `70dcf397dc04470b9756119a9fe8f6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:40:57.372690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:57.379466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1413/2790] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:40:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:40:57.995402 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:40:57.997237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:58.078564 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247062a8-e609-49f0-a134-eea2e1076b71 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:40:58.083757 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:40:58.084385 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:40:58.084663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:40:58.084931 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:40:58.086744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:40:58.100449 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:40:58.102038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:40:58.114419 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1378/2791] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:40:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:41:01.066305 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:41:01.068258 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:41:01.081438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5bb71-0c1a-499e-93de-b3da92e968a7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:41:01.085021 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:41:01.085628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:41:01.085903 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:41:01.086165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:41:01.087908 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:41:01.099153 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:41:01.100624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:41:01.110858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1414/2792] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:41:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:41:07.927311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:41:07.936729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b2500-215e-4700-b619-5ab85ca66fe6 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:41:07.940247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:41:07.940828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:41:07.941100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:41:07.941365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:41:07.943036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:41:08.004007 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:41:08.005413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:41:08.014863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1379/2793] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:41:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:41:11.106351 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=38 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:41:18.010563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:42:41.142083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:41.159618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a0db6-464e-427a-8c23-b93112828eb1 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:41.166905 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:41.167582 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:41.167861 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:41.168130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:41.171400 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:41.193907 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:41.196424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:41.228507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1415/2794] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:42:47.290365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:47.299373 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a0b7b-3e94-440d-94dd-96170c668a0c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:47.303090 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:47.303717 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:47.303999 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:47.304274 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:47.306014 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:47.368391 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:47.369812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:47.379422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1380/2795] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:42:50.682911 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:50.687357 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:50.702862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0dcdf-5ba4-4335-8b64-53d2e9334d0c service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:50.708055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:50.708706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:50.709046 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:50.709384 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] PATH_INFO: `/v3/users/70dcf397dc04470b9756119a9fe8f6a6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:50.710587 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70dcf397dc04470b9756119a9fe8f6a6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:50.712598 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:50.762270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70dcf397dc04470b9756119a9fe8f6a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:50.762957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70dcf397dc04470b9756119a9fe8f6a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:50.769969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70dcf397dc04470b9756119a9fe8f6a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:50.781689 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70dcf397dc04470b9756119a9fe8f6a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:50.782415 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-50bf4033-ed3f-4aaa-8f20-0762a565634f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70dcf397dc04470b9756119a9fe8f6a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:50.795978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1416/2796] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:42:50 2022] DELETE /identity/v3/users/70dcf397dc04470b9756119a9fe8f6a6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:42:50.970744 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:50.981660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0c79-9915-4180-a7de-0dc22d491724 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.039973 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.040599 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.040876 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.041146 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] PATH_INFO: `/v3/projects/7cca63adb761474d82636997dade783a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.042248 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cca63adb761474d82636997dade783a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.044151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.089321 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cca63adb761474d82636997dade783a', 'request_id': 'req-186970b3-21e5-42db-a3cd-a88af92e7300'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:51.095906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cca63adb761474d82636997dade783a', 'request_id': 'req-186970b3-21e5-42db-a3cd-a88af92e7300'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:51.103266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cca63adb761474d82636997dade783a', 'request_id': 'req-186970b3-21e5-42db-a3cd-a88af92e7300'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:51.114547 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] The token cache is being invalidate because project 7cca63adb761474d82636997dade783a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:42:51.114811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:51.115455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-186970b3-21e5-42db-a3cd-a88af92e7300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:42:51.117015 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1381/2797] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:42:50 2022] DELETE /identity/v3/projects/7cca63adb761474d82636997dade783a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:42:51.121996 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.122285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.122558 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.142269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:42:51.143461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:51.202529 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1417/2798] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:42:51.210950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.211235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.211517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.231880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:42:51.233070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:51.240884 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1382/2799] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:42:51.244949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.254783 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e7bcf-b34f-4dd3-aba1-2c74978b2a09 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.259316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.259944 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.260225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.260498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.261138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:42:51.261433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.263011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.274918 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1418/2800] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:42:51.279236 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.289207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d3e69-e5eb-483a-9d4b-6782cb297afb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.293820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.294405 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.294693 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.294951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.295867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.297770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.313190 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:42:51.327032 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1383/2801] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:42:51 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:42:51.331385 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.341858 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082625-f7a1-4990-8e11-4412ef827138 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.386581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:42:51.401084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.401678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.401957 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.402228 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.402853 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.404794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.425277 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1419/2802] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:42:51 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:42:51.429702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.439768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe06b4-6da1-40fd-a934-fb91d6961a54 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.445185 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.445769 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.446045 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.446312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.446944 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.448551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.458813 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1384/2803] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:42:51.462711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.473423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b744e-adb9-4ae9-8f2e-4b0aa3923798 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.477988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.478569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.478846 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.479118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.479785 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.481354 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e8f24-b779-4962-a9b3-263b0f77e256 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.492020 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1420/2804] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:42:51.499812 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.512180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9feb4a-a260-453c-a6b1-6dfe3681b629 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.517062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.517741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.518022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.518295 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] PATH_INFO: `/v3/projects/147f589a02b0462fb1641e43acb254fb/users/04a1343b5f7640f692d358496a1d6e01/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.533375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=147f589a02b0462fb1641e43acb254fb, user_id=04a1343b5f7640f692d358496a1d6e01, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.536135 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.549113 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1385/2805] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:42:51 2022] PUT /identity/v3/projects/147f589a02b0462fb1641e43acb254fb/users/04a1343b5f7640f692d358496a1d6e01/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:42:51.560187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.560476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.560756 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.590870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] MFA Rules not processed for user `04a1343b5f7640f692d358496a1d6e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:42:51.592109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:51.654765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1421/2806] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:42:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:42:51.666144 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:42:51.667880 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:42:51.741893 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4b41b-50f5-45b2-9f0b-a43a62c1f7da admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:42:51.747388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:42:51.748008 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:42:51.748290 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:42:51.748563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:42:51.757381 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:42:51.758942 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb5d0-d3e3-4bb5-8177-b45835b2c81d service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:42:51.767802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1386/2807] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:42:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:01.156202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:01.214509 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799f49d-8eb3-42eb-82bb-7dfc1bf8c41c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:01.219028 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:01.219779 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.219995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.220261 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] PATH_INFO: `/v3/users/04a1343b5f7640f692d358496a1d6e01` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.221360 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a1343b5f7640f692d358496a1d6e01)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:01.223256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:01.268737 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a1343b5f7640f692d358496a1d6e01'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.269405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a1343b5f7640f692d358496a1d6e01'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.276771 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a1343b5f7640f692d358496a1d6e01'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.277899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:01.290371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a1343b5f7640f692d358496a1d6e01'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.291006 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a1343b5f7640f692d358496a1d6e01'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.303499 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1422/2808] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:01 2022] DELETE /identity/v3/users/04a1343b5f7640f692d358496a1d6e01 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:01.308017 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.308513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.308629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.327739 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:01.328948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:01.387106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1387/2809] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:01.397480 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:01.398909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:01.455371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70fa7-141d-4d36-91be-027a50b0d2c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:01.459751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:01.460348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.460628 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.460901 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.469874 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:01.471232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ff126-354e-4dab-989b-d327ced69214 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:01.480995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1423/2810] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:01.747664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:01.749084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:01.761570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d44e9-3e00-40ac-a916-fd650eb0eac0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:01.766320 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:01.767091 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.767375 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.767678 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] PATH_INFO: `/v3/projects/147f589a02b0462fb1641e43acb254fb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.768851 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147f589a02b0462fb1641e43acb254fb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:01.770759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:01.818700 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:01.826283 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147f589a02b0462fb1641e43acb254fb', 'request_id': 'req-41735451-b358-4fd5-8732-9e2a34ee1622'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.834037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147f589a02b0462fb1641e43acb254fb', 'request_id': 'req-41735451-b358-4fd5-8732-9e2a34ee1622'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.840427 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147f589a02b0462fb1641e43acb254fb', 'request_id': 'req-41735451-b358-4fd5-8732-9e2a34ee1622'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.851313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] The token cache is being invalidate because project 147f589a02b0462fb1641e43acb254fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:01.851629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.852250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-41735451-b358-4fd5-8732-9e2a34ee1622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:01.853709 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1388/2811] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:01 2022] DELETE /identity/v3/projects/147f589a02b0462fb1641e43acb254fb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:01.858551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.858841 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.859120 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.878920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:01.880391 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:01.939670 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1424/2812] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:01.947791 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.948077 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.948349 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.967822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:01.969020 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:01.977695 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1389/2813] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:01.981749 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:01.991970 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8c5b-3b36-4669-9264-17f0f4924653 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:01.996459 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:01.997106 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:01.997362 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:01.997635 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:01.998280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:43:01.998581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.000225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.011655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1425/2814] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:43:02.015740 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.025358 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f2955-e437-41ba-9801-0ea39aa02de8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.029974 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.030593 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.030869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.031139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.031803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.034138 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.062098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1390/2815] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:02.066327 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.076361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed07395-1f76-496e-99b0-5761a5adb4ce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.131718 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.132320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.132601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.132873 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.133508 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.135397 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.145176 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:02.158566 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1426/2816] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:43:02.163490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.174027 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588a48-a54d-4f49-95c4-91cbdb6954e7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.179664 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.180261 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.180540 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.180810 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.181478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.183051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.192970 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1391/2817] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:02.196973 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.206752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ebcea-4d08-4b5c-9a92-3bb8f5b211ec admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.211513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.212128 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.212409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.212679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.213317 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.214882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57d1d-8e83-4692-a47b-6745a49ec6dc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.225829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1427/2818] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:02.229773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.240662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40038-b7b0-4546-bbf9-69df7a859c4d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.245346 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.245925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.246205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.246483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] PATH_INFO: `/v3/projects/c76fd8eccbb94e9d95d1a49e3a3be187/users/7346bd8035694c73bf8cd696fa9be5d5/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.261051 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76fd8eccbb94e9d95d1a49e3a3be187, user_id=7346bd8035694c73bf8cd696fa9be5d5, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.263687 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.276799 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1392/2819] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:02 2022] PUT /identity/v3/projects/c76fd8eccbb94e9d95d1a49e3a3be187/users/7346bd8035694c73bf8cd696fa9be5d5/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:02.289388 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.289674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.289949 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.317044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] MFA Rules not processed for user `7346bd8035694c73bf8cd696fa9be5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:02.318290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:02.380312 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1428/2820] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:02.384919 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.446333 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea542b8-123b-4eb6-ab46-d208ae7e4226 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.450927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.451525 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.451833 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.452104 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.452728 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.454600 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.483187 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1393/2821] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:02.487113 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.497711 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85457c06-d6ba-4882-94cc-319acb7d4ad4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.556742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.557338 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.557616 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.557887 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.558511 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.560485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.581541 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1429/2822] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:02.587512 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.601468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbdd53-850c-4790-b6c8-cc735aec0191 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.606176 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.606770 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.607054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.607330 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.608005 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.609597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.619927 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1394/2823] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:02.624240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.635158 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac17088-c713-4766-85cd-728d84011eaf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.639763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.640420 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.640720 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.640989 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] PATH_INFO: `/v3/projects/2008dc166e9e4a00ac14a42daeaa1817/users/a08fa6e5128c4b9da9eda2f6c9785890/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.656045 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2008dc166e9e4a00ac14a42daeaa1817, user_id=a08fa6e5128c4b9da9eda2f6c9785890, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.658647 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.671815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1430/2824] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:02 2022] PUT /identity/v3/projects/2008dc166e9e4a00ac14a42daeaa1817/users/a08fa6e5128c4b9da9eda2f6c9785890/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:43:02.675759 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.686191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac4c58-32cc-4a3e-a95a-c39b043493e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.724458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:02.744653 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.745254 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.745533 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.745803 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.746455 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.748160 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c492d-819a-4f3c-8da2-e76935a9dfe6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.759207 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1395/2825] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:02.763521 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.774770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b6b-0006-4202-a974-4164833308c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.779368 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.780039 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.780320 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.780596 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] PATH_INFO: `/v3/projects/2008dc166e9e4a00ac14a42daeaa1817/users/a08fa6e5128c4b9da9eda2f6c9785890/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.782493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2008dc166e9e4a00ac14a42daeaa1817, user_id=a08fa6e5128c4b9da9eda2f6c9785890, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.785137 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:02.795345 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1431/2826] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:02 2022] PUT /identity/v3/projects/2008dc166e9e4a00ac14a42daeaa1817/users/a08fa6e5128c4b9da9eda2f6c9785890/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:02.806741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.807080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.807359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.836272 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] MFA Rules not processed for user `a08fa6e5128c4b9da9eda2f6c9785890`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:02.837478 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:02.904180 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1396/2827] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:02.915526 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:02.917268 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:02.945270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:02.976034 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7deaa-f755-4505-8094-33933091a663 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:02.979564 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:02.980225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:02.980504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:02.980775 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:02.989997 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:02.991409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb370e27-062f-4b57-8e7c-fb81609982e5 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:03.000602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1432/2828] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:03.424316 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:03.484984 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f626a-2555-43a3-bcfa-1ccd421c2ad8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:03.490111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:03.490729 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:03.491010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:03.491281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] PATH_INFO: `/v3/users/7346bd8035694c73bf8cd696fa9be5d5` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:03.492435 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7346bd8035694c73bf8cd696fa9be5d5)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:03.494332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:03.538576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7346bd8035694c73bf8cd696fa9be5d5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:03.539139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7346bd8035694c73bf8cd696fa9be5d5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:03.546062 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7346bd8035694c73bf8cd696fa9be5d5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:03.551462 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:03.560065 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7346bd8035694c73bf8cd696fa9be5d5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:03.560702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7346bd8035694c73bf8cd696fa9be5d5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:03.572792 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1397/2829] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:03 2022] DELETE /identity/v3/users/7346bd8035694c73bf8cd696fa9be5d5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:03.577363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:03.577847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:03.577966 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:03.596798 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:03.597978 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:03.658522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1433/2830] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:03.668848 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:03.670235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:03.728061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac80700-7770-4ef2-82e3-d271fe453d72 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:03.731641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:03.732233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:03.732517 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:03.732774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:03.742281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:03.743727 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:03.752887 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1398/2831] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:04.034076 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.035713 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:04.048382 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55333c99-f604-47c9-a438-d5b86fe53626 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:04.053419 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:04.054044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.054318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.054588 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] PATH_INFO: `/v3/projects/c76fd8eccbb94e9d95d1a49e3a3be187` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.055731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76fd8eccbb94e9d95d1a49e3a3be187)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:04.057621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:04.104429 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76fd8eccbb94e9d95d1a49e3a3be187', 'request_id': 'req-2be56471-42f9-4220-92be-cec2d6a0cb02'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.111318 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76fd8eccbb94e9d95d1a49e3a3be187', 'request_id': 'req-2be56471-42f9-4220-92be-cec2d6a0cb02'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.116980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76fd8eccbb94e9d95d1a49e3a3be187', 'request_id': 'req-2be56471-42f9-4220-92be-cec2d6a0cb02'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.127876 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] The token cache is being invalidate because project c76fd8eccbb94e9d95d1a49e3a3be187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:04.128160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.128781 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.131029 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1434/2832] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:04 2022] DELETE /identity/v3/projects/c76fd8eccbb94e9d95d1a49e3a3be187 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:04.135204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.144703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f731572-fbf6-45e5-8c49-7572d48a444a service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:04.201403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:04.202024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.202300 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.202570 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] PATH_INFO: `/v3/users/a08fa6e5128c4b9da9eda2f6c9785890` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.203703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08fa6e5128c4b9da9eda2f6c9785890)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:04.205595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:04.250224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08fa6e5128c4b9da9eda2f6c9785890'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.250856 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08fa6e5128c4b9da9eda2f6c9785890'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.258364 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08fa6e5128c4b9da9eda2f6c9785890'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.269996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08fa6e5128c4b9da9eda2f6c9785890'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.271237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1a63dd83-faba-40da-882f-e1910463dae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08fa6e5128c4b9da9eda2f6c9785890'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.283807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1399/2833] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:04 2022] DELETE /identity/v3/users/a08fa6e5128c4b9da9eda2f6c9785890 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:04.579313 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.580815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:04.594101 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56471-42f9-4220-92be-cec2d6a0cb02 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:04.644466 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:04.656124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:04.656759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.657040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.657515 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] PATH_INFO: `/v3/projects/2008dc166e9e4a00ac14a42daeaa1817` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.658726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2008dc166e9e4a00ac14a42daeaa1817)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:04.660925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:04.709958 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2008dc166e9e4a00ac14a42daeaa1817', 'request_id': 'req-c348e3d2-239a-451a-a0a4-a9d45bd02758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.718413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2008dc166e9e4a00ac14a42daeaa1817', 'request_id': 'req-c348e3d2-239a-451a-a0a4-a9d45bd02758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.725353 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2008dc166e9e4a00ac14a42daeaa1817', 'request_id': 'req-c348e3d2-239a-451a-a0a4-a9d45bd02758'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.737590 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] The token cache is being invalidate because project 2008dc166e9e4a00ac14a42daeaa1817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:04.737882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.738556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c348e3d2-239a-451a-a0a4-a9d45bd02758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:04.740557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1435/2834] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:04 2022] DELETE /identity/v3/projects/2008dc166e9e4a00ac14a42daeaa1817 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:04.745603 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.746107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.746217 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.753754 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:04.768296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:04.769505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:04.829050 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1400/2835] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:04.836947 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.837229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.837497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.857536 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:04.858730 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:04.867171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1436/2836] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:04.871293 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.880867 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0241c-7e8b-4520-b3be-4dc72def69a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:04.885421 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:04.885948 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.886224 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.886487 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.887127 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:43:04.887423 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:04.889042 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:04.900322 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1401/2837] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:43:04.904411 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.914390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b22ade-6752-46b1-bfb7-8eba4555da34 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:04.919011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:04.919632 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:04.919913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:04.920184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:04.921071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:04.922953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:04.950231 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1437/2838] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:04 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:04.954458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:04.964553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0c3-daa4-4be5-90f7-899cb545b205 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.019222 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.019914 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.020191 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.020458 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.021220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.023101 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.043921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1402/2839] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:04 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:05.048405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.059474 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef4191-a02b-4866-9424-cc75461b4b71 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.064077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.064663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.064937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.065204 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.065834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.067428 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.077302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1438/2840] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:05.081048 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.091152 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33bfee-c47c-4cbf-b1f6-9ab8b524959f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.095696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.096276 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.096549 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.096815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.097446 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.099648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992c111-59c9-4d3c-9345-b4e6ebcd794a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.109967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1403/2841] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:05.113889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.125482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8fc10-ea01-41a4-af99-60c4b496d1fc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.130118 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.130757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.131036 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.131340 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] PATH_INFO: `/v3/projects/31f86c58673949dab1862e1190685561/users/dfa520a3916346b49d25f0df579b6743/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.146229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f86c58673949dab1862e1190685561, user_id=dfa520a3916346b49d25f0df579b6743, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.148912 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.163225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1439/2842] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:05 2022] PUT /identity/v3/projects/31f86c58673949dab1862e1190685561/users/dfa520a3916346b49d25f0df579b6743/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:05.227174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.227450 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.227753 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.255489 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] MFA Rules not processed for user `dfa520a3916346b49d25f0df579b6743`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:05.256712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:05.315565 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:05.323556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1404/2843] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:05.328093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.389291 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735a4ef-6711-47de-bfa8-d88c7800b3d1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.393967 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.394497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.394774 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.395043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.396245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.398156 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.424844 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1440/2844] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:05 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:05.429166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.440069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbd3b1-46db-49b3-b5eb-e39c2bbed144 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.496486 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.497094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.497403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.497720 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.498337 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.500306 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.521022 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1405/2845] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:05 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:05.525004 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.530512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:05.538124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f19d74-9ef7-4a60-bb6f-88072e9076cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.544124 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.544739 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.545042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.545299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.546192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.547814 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.558601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1441/2846] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:05.562705 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.573513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df861ed-3aec-4b73-b6e7-c7aa5063264c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.578122 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.578764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.579041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.579314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] PATH_INFO: `/v3/projects/6d410c1de14b4a7c89abfbefdae309d7/users/c87a67787ee24a57901cc336f2987ce6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.594646 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d410c1de14b4a7c89abfbefdae309d7, user_id=c87a67787ee24a57901cc336f2987ce6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.597315 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.611099 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1406/2847] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:05 2022] PUT /identity/v3/projects/6d410c1de14b4a7c89abfbefdae309d7/users/c87a67787ee24a57901cc336f2987ce6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:05.615187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.625062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4fc05-8112-48e8-8609-f88ed299f223 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.683556 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.684203 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.684555 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.685051 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.685742 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.687494 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba18769-0ebf-48e0-be1a-89ff8c37c102 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.699424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1442/2848] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:05.703775 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.714667 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119be7b-e4dc-4419-935a-ce33b4e19137 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.719528 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.720199 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.720483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.720758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] PATH_INFO: `/v3/projects/6d410c1de14b4a7c89abfbefdae309d7/users/c87a67787ee24a57901cc336f2987ce6/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.722691 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d410c1de14b4a7c89abfbefdae309d7, user_id=c87a67787ee24a57901cc336f2987ce6, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.725379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.735654 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1407/2849] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:05 2022] PUT /identity/v3/projects/6d410c1de14b4a7c89abfbefdae309d7/users/c87a67787ee24a57901cc336f2987ce6/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:05.746130 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.746442 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.746716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.774893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] MFA Rules not processed for user `c87a67787ee24a57901cc336f2987ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:05.776102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:05.845988 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1443/2850] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:05.857220 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:05.858959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:05.917111 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881d83-7d34-4d64-b85f-d0f6a522861c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:05.920694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:05.921537 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:05.921811 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:05.922078 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:05.930963 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:05.932396 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:05.941641 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1408/2851] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:06.191544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:06.201867 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd81c4-34a0-4d5c-89a3-44b56adf0416 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:06.205339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:06.205925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:06.206201 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:06.206475 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:06.261302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:06.262734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeb19e-6314-4e34-97d8-0c34852bfdb6 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:06.271614 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1444/2852] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:08.996568 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:09.049064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:09.061553 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4a4a1-8553-42e3-9055-21cb3109cfd1 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:09.066139 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:09.066764 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:09.067040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:09.067350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] PATH_INFO: `/v3/users/dfa520a3916346b49d25f0df579b6743` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:09.068554 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa520a3916346b49d25f0df579b6743)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:09.070490 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:09.115120 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa520a3916346b49d25f0df579b6743'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.115815 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa520a3916346b49d25f0df579b6743'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.122660 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa520a3916346b49d25f0df579b6743'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.134594 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa520a3916346b49d25f0df579b6743'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.135168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa520a3916346b49d25f0df579b6743'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.147831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1409/2853] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:08 2022] DELETE /identity/v3/users/dfa520a3916346b49d25f0df579b6743 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:09.152617 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:09.153210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:09.153210 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:09.174133 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:09.175352 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:09.180752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:09.235671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1445/2854] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:09.245560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:09.246951 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:09.303969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84521c-b45a-4da0-9ff5-adc91254ffca admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:09.307620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:09.308218 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:09.308498 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:09.308768 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:09.317783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:09.319847 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:09.329142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1410/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:09.591365 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:09.592812 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:09.605000 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7f4c6-6e6e-4f8a-b125-04b666e92090 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:09.609551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:09.610178 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:09.610457 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:09.610728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] PATH_INFO: `/v3/projects/31f86c58673949dab1862e1190685561` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:09.611881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f86c58673949dab1862e1190685561)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:09.613757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:09.667723 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f86c58673949dab1862e1190685561', 'request_id': 'req-49d38898-fa18-4222-821b-0596cffedc3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.674826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f86c58673949dab1862e1190685561', 'request_id': 'req-49d38898-fa18-4222-821b-0596cffedc3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.680431 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f86c58673949dab1862e1190685561', 'request_id': 'req-49d38898-fa18-4222-821b-0596cffedc3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.691806 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] The token cache is being invalidate because project 31f86c58673949dab1862e1190685561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:09.692093 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.692767 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.694953 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1446/2856] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:09 2022] DELETE /identity/v3/projects/31f86c58673949dab1862e1190685561 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:09.699121 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:09.710178 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd289cf-bf89-436b-9a24-4fed02a668a4 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:09.767632 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:09.768251 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:09.768530 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:09.768802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] PATH_INFO: `/v3/users/c87a67787ee24a57901cc336f2987ce6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:09.769903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87a67787ee24a57901cc336f2987ce6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:09.772037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:09.787823 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:09.821036 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87a67787ee24a57901cc336f2987ce6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.821672 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87a67787ee24a57901cc336f2987ce6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.828456 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87a67787ee24a57901cc336f2987ce6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.840229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87a67787ee24a57901cc336f2987ce6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.840853 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-fe780c91-b27d-4358-8909-ee02d5fadea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87a67787ee24a57901cc336f2987ce6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:09.853268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1411/2857] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:09 2022] DELETE /identity/v3/users/c87a67787ee24a57901cc336f2987ce6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:43:10.136765 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.138222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:10.155437 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d38898-fa18-4222-821b-0596cffedc3c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.213177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.214077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.214552 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.214829 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] PATH_INFO: `/v3/projects/6d410c1de14b4a7c89abfbefdae309d7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.216008 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d410c1de14b4a7c89abfbefdae309d7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.217913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.256184 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:10.266557 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d410c1de14b4a7c89abfbefdae309d7', 'request_id': 'req-d2244ed1-247d-4c45-829d-ad0aa2e9600f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:10.274011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d410c1de14b4a7c89abfbefdae309d7', 'request_id': 'req-d2244ed1-247d-4c45-829d-ad0aa2e9600f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:10.279696 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d410c1de14b4a7c89abfbefdae309d7', 'request_id': 'req-d2244ed1-247d-4c45-829d-ad0aa2e9600f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:10.291721 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] The token cache is being invalidate because project 6d410c1de14b4a7c89abfbefdae309d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:10.292011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:10.292649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d2244ed1-247d-4c45-829d-ad0aa2e9600f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:10.294111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1447/2858] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:10 2022] DELETE /identity/v3/projects/6d410c1de14b4a7c89abfbefdae309d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:10.299379 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.299913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.300031 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.320197 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:10.321595 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:10.379319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1412/2859] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:10.387198 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.387481 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.387800 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.407171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:10.408727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:10.417551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1448/2860] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:10.422100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.432991 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e77b20-076a-47e4-9a87-f761cc171f21 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.437571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.438162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.438437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.438703 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.439339 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:43:10.439665 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.441237 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.452774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1413/2861] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:43:10.457639 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.467752 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a01ab-e0d5-4512-a358-c43de514030d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.472296 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.472882 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.473160 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.473430 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.474052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.476003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.503232 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1449/2862] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:10.507392 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.517625 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782a52-2c40-410a-94c4-7fa3d0484d23 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.574262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.574872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.575151 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.575425 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.576089 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.577986 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.600312 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1414/2863] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:10.605334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.616191 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65157d3-ad9d-45ea-b744-d412fcfff0a4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.621092 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.621688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.621968 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.622240 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.622881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.624507 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.638875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1450/2864] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:10.640083 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.651171 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6dd22-8107-49d7-ba83-d665a1a2b40e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.655776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.656377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.656656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.656937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.657579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.659163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb3400-93dc-4940-8819-4c2f9bb101ce admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.665556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:10.672704 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1415/2865] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:43:10.676551 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.687052 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fb551-72e6-4f28-aaec-780f0cd7b9e0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.691615 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.692256 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.692537 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.692810 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] PATH_INFO: `/v3/projects/ac6c9c61594c4159b33381bb256c7755/users/126e5a3701be45e68451627f247f8757/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.707424 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6c9c61594c4159b33381bb256c7755, user_id=126e5a3701be45e68451627f247f8757, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.710070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.723476 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1451/2866] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:10 2022] PUT /identity/v3/projects/ac6c9c61594c4159b33381bb256c7755/users/126e5a3701be45e68451627f247f8757/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:10.735776 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.736200 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.736545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.764631 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] MFA Rules not processed for user `126e5a3701be45e68451627f247f8757`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:10.765846 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:10.830069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1416/2867] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:10.834633 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.894719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cd631-05ac-4734-acb1-e434ab963f3e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:10.900047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:10.900648 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:10.900932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:10.901207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:10.901833 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:10.903744 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:10.931413 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1452/2868] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:10.935514 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:10.946829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9e1b1-f5d5-4a90-826c-979893abc661 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.005194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.005719 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.005998 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.006268 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.006908 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.009796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.033239 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1417/2869] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:10 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:11.037309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.048637 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae783f2-495b-4069-8b2f-83619694dc0b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.053202 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.053790 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.054071 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.054348 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.054984 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.056838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.067175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1453/2870] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:11.072071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.082023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a293ff-651a-4ffb-90f7-e1ccad89c4e2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.087413 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.088096 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.088377 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.088648 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] PATH_INFO: `/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3/users/beada086a214418980f5bd0ed9294052/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.104179 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50729b9a0acd4af5a09b3ab25885c2f3, user_id=beada086a214418980f5bd0ed9294052, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.107159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.122438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1418/2871] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:11 2022] PUT /identity/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3/users/beada086a214418980f5bd0ed9294052/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:11.126493 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.137055 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee82d82-b276-452d-93f4-c35b315094d5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.171207 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:11.204315 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.204897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.205179 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.205455 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.206140 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.207768 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06346832-976f-45df-9609-0f485ab820b8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.217932 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1454/2872] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:11.222069 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.232657 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49d5f-db0d-4685-83aa-837b1d74b142 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.237326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.237983 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.238269 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.238545 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] PATH_INFO: `/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3/users/beada086a214418980f5bd0ed9294052/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.240511 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50729b9a0acd4af5a09b3ab25885c2f3, user_id=beada086a214418980f5bd0ed9294052, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.243859 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.254474 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1419/2873] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:11 2022] PUT /identity/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3/users/beada086a214418980f5bd0ed9294052/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:11.265822 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.266127 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.266440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.294716 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] MFA Rules not processed for user `beada086a214418980f5bd0ed9294052`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:11.296224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:11.369888 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1455/2874] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:11.381054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.383527 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:11.440117 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:11.453989 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b7917-c3ec-4d5f-b7af-2e759864546d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.457640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.458238 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.458515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.458783 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.469054 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.470475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.480130 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1420/2875] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:43:11.568329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:11.580003 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d82297-501c-45ac-b124-ff5172ced5ac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:11.583679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:11.584423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:11.584700 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:11.584969 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:11.645386 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:11.646834 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944acca1-6d46-4ac1-a089-4486577f1847 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:11.655933 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1456/2876] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:16.640213 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:16.741329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213cbec-e085-41b6-b1dd-de11bc355a5f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:16.747736 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:16.748365 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:16.748639 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:16.748907 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] PATH_INFO: `/v3/users/126e5a3701be45e68451627f247f8757` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:16.750026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126e5a3701be45e68451627f247f8757)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:16.751968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:16.799331 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126e5a3701be45e68451627f247f8757'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:16.800040 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126e5a3701be45e68451627f247f8757'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:16.807843 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126e5a3701be45e68451627f247f8757'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:16.824475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126e5a3701be45e68451627f247f8757'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:16.825280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126e5a3701be45e68451627f247f8757'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:16.838966 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1421/2877] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:16 2022] DELETE /identity/v3/users/126e5a3701be45e68451627f247f8757 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:16.843363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:16.843866 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:16.844018 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:16.899875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:16.901801 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:16.928332 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:16.964803 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1457/2878] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:16.974437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:16.975994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:17.001618 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:17.050950 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ae38-5f1c-4a90-a868-a4005a891649 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:17.055250 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:17.055891 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:17.056168 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:17.056438 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:17.065233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:17.066644 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:17.075915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1422/2879] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:17.345870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:17.347325 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:17.360847 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16d759-7c38-48bb-83f1-3679c2fa966a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:17.365524 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:17.366214 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:17.366497 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:17.366792 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] PATH_INFO: `/v3/projects/ac6c9c61594c4159b33381bb256c7755` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:17.368299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6c9c61594c4159b33381bb256c7755)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:17.370219 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:17.417574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6c9c61594c4159b33381bb256c7755', 'request_id': 'req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.424899 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6c9c61594c4159b33381bb256c7755', 'request_id': 'req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.430794 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6c9c61594c4159b33381bb256c7755', 'request_id': 'req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.442157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] The token cache is being invalidate because project ac6c9c61594c4159b33381bb256c7755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:17.442441 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.443069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.444609 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1458/2880] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:17 2022] DELETE /identity/v3/projects/ac6c9c61594c4159b33381bb256c7755 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:17.448987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:17.458563 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ca2e-af79-4f97-9b6c-71cd1c946e2c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:17.516656 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:17.517284 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:17.517560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:17.517831 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] PATH_INFO: `/v3/users/beada086a214418980f5bd0ed9294052` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:17.518925 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beada086a214418980f5bd0ed9294052)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:17.521206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:17.568725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beada086a214418980f5bd0ed9294052'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.569356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beada086a214418980f5bd0ed9294052'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.576899 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beada086a214418980f5bd0ed9294052'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.588849 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beada086a214418980f5bd0ed9294052'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.589579 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-151cc42c-ab9c-48a8-9073-f5ce9a9b3d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beada086a214418980f5bd0ed9294052'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:17.603469 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1423/2881] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:17 2022] DELETE /identity/v3/users/beada086a214418980f5bd0ed9294052 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:17.871083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:17.872514 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:17.886674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe2e05-3346-4ca5-afcb-2e4e1d011f24 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:17.945533 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:17.946168 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:17.946433 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:17.946697 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] PATH_INFO: `/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:17.947854 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=50729b9a0acd4af5a09b3ab25885c2f3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:17.949902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:17.997571 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50729b9a0acd4af5a09b3ab25885c2f3', 'request_id': 'req-4214bb3c-88a2-4b67-924b-cab8a262a38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:18.001371 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:18.006461 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50729b9a0acd4af5a09b3ab25885c2f3', 'request_id': 'req-4214bb3c-88a2-4b67-924b-cab8a262a38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:18.014734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50729b9a0acd4af5a09b3ab25885c2f3', 'request_id': 'req-4214bb3c-88a2-4b67-924b-cab8a262a38a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:18.026438 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] The token cache is being invalidate because project 50729b9a0acd4af5a09b3ab25885c2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:18.026731 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:18.027392 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4214bb3c-88a2-4b67-924b-cab8a262a38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:18.028909 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1459/2882] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:17 2022] DELETE /identity/v3/projects/50729b9a0acd4af5a09b3ab25885c2f3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:18.034662 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.035206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.035206 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.076166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:18.088057 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:18.089837 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:18.159888 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1424/2883] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:18.169705 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.169998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.170271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.191042 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:18.192269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:18.201068 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1460/2884] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:18.205282 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.215305 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eb031-6a49-42d7-838e-eb4ec73e4be0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.219877 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.220468 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.220743 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.221010 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.221940 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:43:18.222235 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.223841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.235997 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1425/2885] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 21:43:18.240649 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.251139 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa844a-27e8-42a0-980f-10810da9fe36 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.255855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.256448 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.256726 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.256993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.258280 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.260200 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.288491 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1461/2886] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:18.293472 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.304107 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430f0f5-ef6f-4743-98b5-233e4446b903 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.362481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.363082 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.363359 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.363661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.364324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.366227 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.393559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1426/2887] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:18.395069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.405863 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e46ea-1824-4d90-8bc8-e6f934c76a13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.411286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.412383 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.412924 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.413217 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.414599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.416263 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.428780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1462/2888] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:18.433582 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.443987 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66620b5-630c-4e52-ac1b-6fe7c94a3821 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.448953 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.449551 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.449834 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.450106 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.450758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.452384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b707a-c3f8-4809-a22d-9b5c4f374aec admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.463566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1427/2889] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:43:18.467751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.479047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0947-3f18-4197-a695-7e14e255f23a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.483747 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.484403 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.484687 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.484964 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] PATH_INFO: `/v3/projects/d4a4273459dc45079c86bbae69ffccbb/users/620705b4b278493b8210efb55596a666/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.500502 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a4273459dc45079c86bbae69ffccbb, user_id=620705b4b278493b8210efb55596a666, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.503149 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.516835 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1463/2890] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:18 2022] PUT /identity/v3/projects/d4a4273459dc45079c86bbae69ffccbb/users/620705b4b278493b8210efb55596a666/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:18.527420 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.527796 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.528162 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.557311 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] MFA Rules not processed for user `620705b4b278493b8210efb55596a666`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:18.558607 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:18.622605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:18.631146 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.631681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1428/2891] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:18.705314 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46d10-8e28-450c-aada-cebde343d5ca admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.710062 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.711309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.711751 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.712070 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.712706 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.714602 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.752998 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1464/2892] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:18.754368 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.770645 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e58cff-4374-415d-9bc1-834f72f6a334 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.846929 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.847992 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.848285 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.848560 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.849204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.851773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.879403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1429/2893] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:18 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:18.880895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.901271 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db283-71c6-408b-8a5a-3f4b29cd1567 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.907925 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.908578 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.908864 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.909150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.910087 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.912043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.932363 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1465/2894] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:18.933663 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:18.946855 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f45cc-3ac7-44e3-b937-76d182ad07bb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:18.955864 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:18.957996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:18.959070 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:18.959356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] PATH_INFO: `/v3/projects/7c172d8dd30340a0854ef899890d3a49/users/743c3b3601f74520bc2a5698e0f790ae/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:18.978702 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c172d8dd30340a0854ef899890d3a49, user_id=743c3b3601f74520bc2a5698e0f790ae, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:18.981384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:18.996590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1430/2895] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:18 2022] PUT /identity/v3/projects/7c172d8dd30340a0854ef899890d3a49/users/743c3b3601f74520bc2a5698e0f790ae/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:19.001671 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:19.012568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b7d2b-a9a5-424c-a386-881a647f34c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:19.066229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:19.079482 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:19.080138 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:19.083940 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:19.084225 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:19.085012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:19.087349 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa50d4-6b53-4827-8478-5176a14fd275 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:19.098762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1466/2896] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:19.102897 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:19.113640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a025-d625-4842-9a1d-754b18e11243 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:19.118521 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:19.119205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:19.119819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:19.119819 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] PATH_INFO: `/v3/projects/7c172d8dd30340a0854ef899890d3a49/users/743c3b3601f74520bc2a5698e0f790ae/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:19.122006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c172d8dd30340a0854ef899890d3a49, user_id=743c3b3601f74520bc2a5698e0f790ae, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:19.124390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:19.135865 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1431/2897] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:19 2022] PUT /identity/v3/projects/7c172d8dd30340a0854ef899890d3a49/users/743c3b3601f74520bc2a5698e0f790ae/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:19.148811 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:19.149098 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:19.149435 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:19.187208 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] MFA Rules not processed for user `743c3b3601f74520bc2a5698e0f790ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:19.188304 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:19.269612 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1467/2898] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:19.277863 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:19.279758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:19.352968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403c855-e1fe-4fb8-bdb2-006e6e5b55e8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:19.356938 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:19.357559 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:19.357841 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:19.358116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:19.417751 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:19.419203 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:19.428418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1432/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:43:20.308422 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:20.309895 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:20.388288 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72e4ba-d79f-419d-b344-999cd78532a6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:20.394324 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dac985d2461d4132af309ded8c7ce1a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:20.394931 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:20.395213 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:20.395487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:20.397254 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:20.410009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:20.411640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04ce58-c008-4d15-9a5f-2d70a35cfa30 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:20.423022 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1468/2900] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:21.999324 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:22.009481 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:22.087084 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fde9d-94ff-48b8-a563-c10ff1e5dcb4 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:22.091885 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:22.092524 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:22.092805 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:22.093080 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] PATH_INFO: `/v3/users/620705b4b278493b8210efb55596a666` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:22.094219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=620705b4b278493b8210efb55596a666)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:22.096173 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:22.145132 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620705b4b278493b8210efb55596a666'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.145773 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620705b4b278493b8210efb55596a666'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.156142 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620705b4b278493b8210efb55596a666'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.168906 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620705b4b278493b8210efb55596a666'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.169852 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620705b4b278493b8210efb55596a666'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.183426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1433/2901] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:21 2022] DELETE /identity/v3/users/620705b4b278493b8210efb55596a666 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:22.190675 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:22.191150 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:22.191405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:22.211622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:22.214427 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:22.271309 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:22.280187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1469/2902] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:22.290575 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:22.292071 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:22.350328 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83e0c6-b0e6-4b55-97d9-42b83baee293 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:22.354785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:22.355384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:22.355696 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:22.355969 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:22.365636 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:22.367968 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:22.378403 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1434/2903] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:22.660040 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:22.661522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:22.676192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1afa22-034f-4eb4-a4b7-07e6e77041d7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:22.681568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:22.682451 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:22.682734 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:22.683009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] PATH_INFO: `/v3/projects/d4a4273459dc45079c86bbae69ffccbb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:22.684157 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a4273459dc45079c86bbae69ffccbb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:22.686154 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:22.732719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a4273459dc45079c86bbae69ffccbb', 'request_id': 'req-85cfa5c7-cf00-440c-9266-a53662ecc863'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.739630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a4273459dc45079c86bbae69ffccbb', 'request_id': 'req-85cfa5c7-cf00-440c-9266-a53662ecc863'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.745252 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a4273459dc45079c86bbae69ffccbb', 'request_id': 'req-85cfa5c7-cf00-440c-9266-a53662ecc863'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.756218 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] The token cache is being invalidate because project d4a4273459dc45079c86bbae69ffccbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:22.756512 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.757155 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.758736 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1470/2904] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:22 2022] DELETE /identity/v3/projects/d4a4273459dc45079c86bbae69ffccbb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:22.762874 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:22.772515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444a3e8-32a4-467b-bc61-fb3ae6c58ead service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:22.799116 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:22.832374 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:22.833118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:22.833399 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:22.833670 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] PATH_INFO: `/v3/users/743c3b3601f74520bc2a5698e0f790ae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:22.834802 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743c3b3601f74520bc2a5698e0f790ae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:22.839037 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:22.884428 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743c3b3601f74520bc2a5698e0f790ae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.885061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743c3b3601f74520bc2a5698e0f790ae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.891793 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743c3b3601f74520bc2a5698e0f790ae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.903956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743c3b3601f74520bc2a5698e0f790ae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.904584 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c0919f-cfb4-405c-8ccd-a4e0614c2eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743c3b3601f74520bc2a5698e0f790ae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:22.917223 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1435/2905] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:22 2022] DELETE /identity/v3/users/743c3b3601f74520bc2a5698e0f790ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:23.214346 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.215926 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.237859 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfa5c7-cf00-440c-9266-a53662ecc863 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.299180 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.299837 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.300115 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.300387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] PATH_INFO: `/v3/projects/7c172d8dd30340a0854ef899890d3a49` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.301498 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c172d8dd30340a0854ef899890d3a49)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.303465 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.352727 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c172d8dd30340a0854ef899890d3a49', 'request_id': 'req-fad91ca1-8db1-4ea4-a78d-be07494c07a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:23.360410 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c172d8dd30340a0854ef899890d3a49', 'request_id': 'req-fad91ca1-8db1-4ea4-a78d-be07494c07a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:23.366329 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c172d8dd30340a0854ef899890d3a49', 'request_id': 'req-fad91ca1-8db1-4ea4-a78d-be07494c07a1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:23.379267 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] The token cache is being invalidate because project 7c172d8dd30340a0854ef899890d3a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:23.379569 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:23.380290 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:23.381938 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1471/2906] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:23 2022] DELETE /identity/v3/projects/7c172d8dd30340a0854ef899890d3a49 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:23.387576 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be775-d54a-4e25-8406-b9df4d3668eb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.388212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be775-d54a-4e25-8406-b9df4d3668eb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.388212 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be775-d54a-4e25-8406-b9df4d3668eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.420994 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c42be775-d54a-4e25-8406-b9df4d3668eb None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:23.422712 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c42be775-d54a-4e25-8406-b9df4d3668eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.464871 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.466270 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.480664 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:23.490921 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1436/2907] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:23.499128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.499414 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.499735 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.520194 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:23.521408 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.527825 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad91ca1-8db1-4ea4-a78d-be07494c07a1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.529535 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1437/2908] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:23.531845 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.532454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.532805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.533081 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.533767 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.534826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.543395 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3fd6f-5be6-4737-ac1f-7325206dfbe7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.548426 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.549452 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.549860 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.550041 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.550325 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.550601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.551247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 21:43:23.551577 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.553157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.560690 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1472/2909] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 21:43:23.572694 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1438/2910] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 21:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 21:43:23.574131 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.583772 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd0a4e-5efb-4863-9223-8876894fa3c0 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.588547 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.589135 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.589418 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.589708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.590316 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.592230 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.622828 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1473/2911] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:23.629095 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.643279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256628-7379-4230-ba41-9b9c4fd8d200 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.670371 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:23.705166 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.705777 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.706056 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.706326 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.706982 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.709003 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.733892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1439/2912] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:23.738072 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.749889 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e554c-989c-44db-ae68-613f644d2dd4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.754708 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.755302 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.755608 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.755881 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.756522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.758108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.768471 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1474/2913] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 21:43:23.772471 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.782329 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8ba3f-405e-47b4-a34a-670966c55fd6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.786915 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.787505 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.787825 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.788093 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.789356 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.790913 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ec418-3bd7-4015-93db-e87c70f0c88a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.802081 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1440/2914] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:23.806653 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:23.816572 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1473-b9da-4b0a-943e-a3541ee89d4e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:23.821440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:23.822084 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.822390 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.822661 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] PATH_INFO: `/v3/projects/44231b622c8e49a0a1ee96a78f9b1acf/users/1e2a546597014f1cb294b06a48f48e39/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.838943 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44231b622c8e49a0a1ee96a78f9b1acf, user_id=1e2a546597014f1cb294b06a48f48e39, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:23.841630 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:23.856462 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1475/2915] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:23 2022] PUT /identity/v3/projects/44231b622c8e49a0a1ee96a78f9b1acf/users/1e2a546597014f1cb294b06a48f48e39/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:43:23.867605 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:23.868118 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:23.868426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:23.897245 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] MFA Rules not processed for user `1e2a546597014f1cb294b06a48f48e39`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:23.898501 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:23.963137 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1441/2916] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 21:43:23.967619 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.033678 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab94f93-44e1-4fe4-a39a-6c15dafc8b37 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.038560 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.040050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.040050 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.040321 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.040897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.042831 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.071875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1476/2917] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:23 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:24.075976 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.087733 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdae6c-fdc8-4630-a4ea-36d165ad49b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.145892 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.146502 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.146788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.147060 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.147741 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.150271 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.178310 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1442/2918] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:24 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 21:43:24.179728 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.190773 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f6a8c-dd7b-4bb4-b557-c07e7aff7bc0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.196011 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.196618 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.196897 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.197165 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.197802 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.199387 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.210993 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1477/2919] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:24.215588 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.226954 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e6040-b87f-4137-9b0b-141b35bf5814 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.231676 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.232279 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.232548 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.232820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.233467 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.235076 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110307b-d2c3-498b-a85b-b2b533c8b43c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.248905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1443/2920] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:24.250116 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.260454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10462208-5532-4997-b66d-2b5ce2299558 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.265033 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.265679 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.265962 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.266233 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] PATH_INFO: `/v3/projects/a7516cb5c90b4f45a41c903718295299/users/65a3ba2916934640ba0e2f9dcd060dbf/roles/869347abd4b848d1abcc7f9e6ffc94a9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.281487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7516cb5c90b4f45a41c903718295299, user_id=65a3ba2916934640ba0e2f9dcd060dbf, role_id=869347abd4b848d1abcc7f9e6ffc94a9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.284144 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.297754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1478/2921] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:24 2022] PUT /identity/v3/projects/a7516cb5c90b4f45a41c903718295299/users/65a3ba2916934640ba0e2f9dcd060dbf/roles/869347abd4b848d1abcc7f9e6ffc94a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:24.309024 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.309483 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.309828 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.339788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] MFA Rules not processed for user `65a3ba2916934640ba0e2f9dcd060dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:24.340920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:24.345967 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:24.409541 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1444/2922] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:24.414762 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.426432 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:24.485563 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adc539-7fa3-4309-a693-ee6322b354b5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.491347 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.492171 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.492546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.492913 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.493651 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.495855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.527757 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1479/2923] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 21:43:24 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:24.531920 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.543566 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0000a-eae1-4095-b46e-d1c896422c32 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.609640 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.610256 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.610534 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.610895 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.612134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.613937 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.642369 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1445/2924] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 21:43:24 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 21:43:24.646044 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.659356 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6cd9-127d-4eff-8c05-8933b68c70ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.665663 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.666293 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.666577 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.666856 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.667520 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.669174 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.690920 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1480/2925] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:24.695445 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.707112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725394e9-c2c7-41b6-9b45-27fa07574ead admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.713053 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.713763 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.714043 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.714314 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] PATH_INFO: `/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a/users/f825128877b04d38b2bd0bc478424d81/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.733209 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0df5fa284348d29c90bd3772d3cc1a, user_id=f825128877b04d38b2bd0bc478424d81, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.736219 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.753327 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1446/2926] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:24 2022] PUT /identity/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a/users/f825128877b04d38b2bd0bc478424d81/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:43:24.758631 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.769740 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6bb0-b7c4-48b4-b145-1c690ace28d0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.828025 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.828644 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.828928 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.829229 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.829916 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.831793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505134e0-0447-485d-937b-44baa4a51dae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.841855 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1481/2927] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 21:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 21:43:24.846011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:24.856597 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371a57-f071-4bc8-9a97-2f4097f20030 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:24.861229 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:24.861820 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.862097 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.862384 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] PATH_INFO: `/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a/users/f825128877b04d38b2bd0bc478424d81/roles/b1aece7abc4d43b78033c71e40fda24e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.864292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0df5fa284348d29c90bd3772d3cc1a, user_id=f825128877b04d38b2bd0bc478424d81, role_id=b1aece7abc4d43b78033c71e40fda24e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:24.867516 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:24.879079 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1447/2928] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 21:43:24 2022] PUT /identity/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a/users/f825128877b04d38b2bd0bc478424d81/roles/b1aece7abc4d43b78033c71e40fda24e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 21:43:24.890286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:24.890574 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:24.890983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:24.918850 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] MFA Rules not processed for user `f825128877b04d38b2bd0bc478424d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:24.920102 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:24.988599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1482/2929] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:25.001956 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:25.003788 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:25.068347 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421cd3f-0d46-4f4f-98a3-c26e6e4b71e0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:25.072204 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:25.072817 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:25.073098 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:25.073370 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:25.083281 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:25.139205 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:25.140809 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:25.152231 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1448/2930] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:26.584181 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:26.593206 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:26.598077 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:26.742809 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ff0e4-f29e-47e7-b4d4-18169ced91fc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:26.749754 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:26.750589 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:26.750942 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:26.751334 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:26.753719 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:26.771399 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:26.773676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:26.789043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1483/2931] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:26.836292 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:26.839026 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:26.853159 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbce-b76f-49ae-a56f-4c18eb18d003 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:26.857280 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:26.857978 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:26.858426 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:26.858758 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:26.916133 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:26.917555 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:26.927006 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1449/2932] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:27.478047 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:27.487599 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac534f4-e212-40ce-900c-6c1e5bb1fbe3 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:27.491111 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b670a7afd2264609894dec2feabc4581', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:27.491732 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:27.492009 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:27.492279 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:27.501478 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:27.502896 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:27.514546 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1484/2933] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:29.934110 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:29.935620 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:30.004745 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3edd7-0f1d-41ad-8973-5b4a56293523 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:30.009905 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:30.010542 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:30.010822 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:30.011094 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] PATH_INFO: `/v3/users/691a25c97f4748119366c35dd4688765` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:30.012262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=691a25c97f4748119366c35dd4688765)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:30.014157 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:30.063262 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691a25c97f4748119366c35dd4688765'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.063924 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691a25c97f4748119366c35dd4688765'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.068756 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:30.076518 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691a25c97f4748119366c35dd4688765'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.093661 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691a25c97f4748119366c35dd4688765'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.094388 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691a25c97f4748119366c35dd4688765'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.107679 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1450/2934] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:29 2022] DELETE /identity/v3/users/691a25c97f4748119366c35dd4688765 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:30.296083 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:30.309134 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db323-963d-46c9-b865-14f870560694 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:30.375692 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:30.376361 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:30.376640 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:30.376937 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] PATH_INFO: `/v3/projects/b9a36910972a4d87b508b918af36bbd5` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:30.378049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a36910972a4d87b508b918af36bbd5)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:30.380043 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:30.430260 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a36910972a4d87b508b918af36bbd5', 'request_id': 'req-e88f4bf6-4754-4bba-8654-60e24f1d89ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.437405 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a36910972a4d87b508b918af36bbd5', 'request_id': 'req-e88f4bf6-4754-4bba-8654-60e24f1d89ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.442674 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:30.444122 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a36910972a4d87b508b918af36bbd5', 'request_id': 'req-e88f4bf6-4754-4bba-8654-60e24f1d89ef'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.456607 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] The token cache is being invalidate because project b9a36910972a4d87b508b918af36bbd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:30.456893 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.457513 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.458955 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1485/2935] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:30 2022] DELETE /identity/v3/projects/b9a36910972a4d87b508b918af36bbd5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:30.463602 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:30.465038 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:30.478681 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54c015-8a95-4405-a201-1a499d7c91f3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:30.536416 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:30.537064 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:30.537344 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:30.537616 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] PATH_INFO: `/v3/users/bb350f44508d427a8b2355822785b8c6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:30.539313 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb350f44508d427a8b2355822785b8c6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:30.541243 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:30.589447 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb350f44508d427a8b2355822785b8c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.590100 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb350f44508d427a8b2355822785b8c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.597443 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb350f44508d427a8b2355822785b8c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.610068 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb350f44508d427a8b2355822785b8c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.610671 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb350f44508d427a8b2355822785b8c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.624390 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1451/2936] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:30 2022] DELETE /identity/v3/users/bb350f44508d427a8b2355822785b8c6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:30.805339 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:30.806770 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:30.820069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4bf6-4754-4bba-8654-60e24f1d89ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:30.878676 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:30.879224 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:30.879504 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:30.879805 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] PATH_INFO: `/v3/projects/cd154266453440d9b564748a7f371c63` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:30.880910 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd154266453440d9b564748a7f371c63)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:30.882832 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:30.928299 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd154266453440d9b564748a7f371c63', 'request_id': 'req-d3839af0-1b96-49b0-a553-7563baa0a897'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.935709 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd154266453440d9b564748a7f371c63', 'request_id': 'req-d3839af0-1b96-49b0-a553-7563baa0a897'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.941222 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd154266453440d9b564748a7f371c63', 'request_id': 'req-d3839af0-1b96-49b0-a553-7563baa0a897'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.952257 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] The token cache is being invalidate because project cd154266453440d9b564748a7f371c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:30.952544 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.953173 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3839af0-1b96-49b0-a553-7563baa0a897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:30.954622 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1486/2937] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:30 2022] DELETE /identity/v3/projects/cd154266453440d9b564748a7f371c63 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:31.479183 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:31.480604 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:31.493829 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9a1c-570d-4856-bd7e-f7829b843d79 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:31.537706 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:31.550840 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:31.551475 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:31.551785 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:31.552061 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] PATH_INFO: `/v3/users/1e2a546597014f1cb294b06a48f48e39` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:31.553174 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2a546597014f1cb294b06a48f48e39)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:31.555629 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:31.602903 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2a546597014f1cb294b06a48f48e39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:31.603571 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2a546597014f1cb294b06a48f48e39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:31.610422 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2a546597014f1cb294b06a48f48e39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:31.623011 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2a546597014f1cb294b06a48f48e39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:31.623713 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2a546597014f1cb294b06a48f48e39'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:31.640109 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1452/2938] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:31 2022] DELETE /identity/v3/users/1e2a546597014f1cb294b06a48f48e39 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:31.644688 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:31.645161 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:31.645281 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:31.664069 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] MFA Rules not processed for user `18ebcbcfc62641d6bab3f3ca3d6448ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 21:43:31.665187 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:31.722936 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1487/2939] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 21:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 21:43:31.732911 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:31.734332 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:31.793578 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46287c6-f497-483a-bbf5-d6ab6f652d55 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:31.797437 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be348541540b47a8acc55e04bbf6adf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:31.798128 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:31.798404 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:31.798807 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:31.808317 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:31.809725 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:31.820513 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1453/2940] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:32.084192 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:32.085804 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:32.098337 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4e05b-b435-4d0b-ad64-78548f6ab66d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:32.103655 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:32.104286 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:32.104568 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:32.104875 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] PATH_INFO: `/v3/projects/44231b622c8e49a0a1ee96a78f9b1acf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:32.106016 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44231b622c8e49a0a1ee96a78f9b1acf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:32.107976 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:32.126275 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:32.157440 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44231b622c8e49a0a1ee96a78f9b1acf', 'request_id': 'req-172870ee-abe9-4f11-82dc-0db9507b6511'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.165112 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44231b622c8e49a0a1ee96a78f9b1acf', 'request_id': 'req-172870ee-abe9-4f11-82dc-0db9507b6511'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.170601 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44231b622c8e49a0a1ee96a78f9b1acf', 'request_id': 'req-172870ee-abe9-4f11-82dc-0db9507b6511'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.182821 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] The token cache is being invalidate because project 44231b622c8e49a0a1ee96a78f9b1acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:32.183108 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.183780 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.185245 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1488/2941] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:32 2022] DELETE /identity/v3/projects/44231b622c8e49a0a1ee96a78f9b1acf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:32.189288 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:32.199590 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b6fb-1717-4e6f-aacf-e3d059c9e7aa service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:32.256247 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:32.256869 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:32.257158 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:32.257436 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] PATH_INFO: `/v3/users/65a3ba2916934640ba0e2f9dcd060dbf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:32.258552 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a3ba2916934640ba0e2f9dcd060dbf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:32.260515 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:32.272350 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:32.307266 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a3ba2916934640ba0e2f9dcd060dbf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.307930 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a3ba2916934640ba0e2f9dcd060dbf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.315134 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a3ba2916934640ba0e2f9dcd060dbf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.327296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a3ba2916934640ba0e2f9dcd060dbf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.328002 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a3ba2916934640ba0e2f9dcd060dbf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.342163 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1454/2942] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:32 2022] DELETE /identity/v3/users/65a3ba2916934640ba0e2f9dcd060dbf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:32.626359 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:32.627995 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:32.642310 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172870ee-abe9-4f11-82dc-0db9507b6511 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:32.711269 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:32.712197 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:32.712487 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:32.712763 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] PATH_INFO: `/v3/projects/a7516cb5c90b4f45a41c903718295299` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:32.713921 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7516cb5c90b4f45a41c903718295299)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:32.715870 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:32.766980 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7516cb5c90b4f45a41c903718295299', 'request_id': 'req-05792454-9373-44f6-a286-f327af1ab3f6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.774729 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7516cb5c90b4f45a41c903718295299', 'request_id': 'req-05792454-9373-44f6-a286-f327af1ab3f6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.780409 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7516cb5c90b4f45a41c903718295299', 'request_id': 'req-05792454-9373-44f6-a286-f327af1ab3f6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.793082 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] The token cache is being invalidate because project a7516cb5c90b4f45a41c903718295299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:32.793372 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.794012 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.795485 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1489/2943] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:32 2022] DELETE /identity/v3/projects/a7516cb5c90b4f45a41c903718295299 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:32.799774 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:32.801155 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:32.813816 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b6b9b-e2ae-47fa-8e8f-b88c47375eac admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:32.869798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:32.870418 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:32.870699 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:32.870971 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] PATH_INFO: `/v3/users/f825128877b04d38b2bd0bc478424d81` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:32.872112 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f825128877b04d38b2bd0bc478424d81)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:32.874023 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:32.921798 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f825128877b04d38b2bd0bc478424d81'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.922465 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f825128877b04d38b2bd0bc478424d81'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.929979 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f825128877b04d38b2bd0bc478424d81'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.941580 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f825128877b04d38b2bd0bc478424d81'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.942216 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f825128877b04d38b2bd0bc478424d81'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:32.955319 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1455/2944] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 21:43:32 2022] DELETE /identity/v3/users/f825128877b04d38b2bd0bc478424d81 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:33.223522 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:33.225095 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:33.238862 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05792454-9373-44f6-a286-f327af1ab3f6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:33.272624 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:33.298621 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ebcbcfc62641d6bab3f3ca3d6448ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc5a8e0f96e744888fe26890d52ecbca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:33.299175 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:33.299454 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:33.299793 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] PATH_INFO: `/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:33.300902 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0df5fa284348d29c90bd3772d3cc1a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:33.302797 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:33.347826 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0df5fa284348d29c90bd3772d3cc1a', 'request_id': 'req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:33.356983 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0df5fa284348d29c90bd3772d3cc1a', 'request_id': 'req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:33.362704 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0df5fa284348d29c90bd3772d3cc1a', 'request_id': 'req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:33.374610 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] The token cache is being invalidate because project bb0df5fa284348d29c90bd3772d3cc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 21:43:33.375177 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:33.375838 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 21:43:33.377319 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1490/2945] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 21:43:33 2022] DELETE /identity/v3/projects/bb0df5fa284348d29c90bd3772d3cc1a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 21:43:38.621872 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:43:38.623273 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:38.635790 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:38.697688 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d4763-e0ba-4289-82e5-9fc94e6ab05e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:43:38.755035 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:43:38.755675 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:43:38.755959 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:43:38.756233 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:43:38.758556 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:43:38.769996 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:43:38.771601 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6e48-b7fd-43dd-b394-ad730f846355 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:43:38.782296 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1456/2946] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:43:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:43:43.371581 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:43:48.778044 dv-cinder-852210-15-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=54 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 21:48:45.638759 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 21:48:45.640852 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 21:48:45.656259 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f2eb1-77c9-44e7-ba6b-b26b67bf10bd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 21:48:45.661212 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ac80d1c221e483989f1e4f5d41a0897', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f864ecf3c22841c98fde56ead0bbe826', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 21:48:45.661815 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 21:48:45.662097 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 21:48:45.662375 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 21:48:45.671423 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 21:48:45.673109 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6ef0c-1a6b-4322-a076-2896b1e72b31 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 21:48:45.683285 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1491/2947] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 21:48:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 21:48:55.679049 dv-cinder-852210-15-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}